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LI. INTRODUCTION AND BACKGROUND 
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Numerical weather prediction models have made great 
progress in forecasting the basic meteorological variables 
and fields on the synoptic scale, such as sea-level pressure, 
wind, and moisture (e.g., relative humidity). However, 
dynamic models have had little success in predicting sensi- 
ble weather variables at the regional/local scale, and in 
fact most models do not forecast many of these variables 
directly. Stochastic-dynamic prediction is being explored 
1t shows promise for operational use sometime in the future, 
but it awaits much further development and more powerful 
computers. 

One of the most significant developments in weather pre- 
diction is the combination of dynamical and statistreas 
methods, known as model output statistics (MOS). The MOS 
technigue is the determination of a statistical relationship 
between a weather element of interest (e.g., visiblity, 
ceiling, precipitation) and a large menu of parameters output 
from an operational numerical prediction model (e.g., boundary 
layer wind, constant-pressure height, temperature). In the 
case of the National Weather Service, the operational MOS 
technique is based on multiple linear regression, where the 
prediction equations are developed from forecast model 


parameters (predictors) and observed weather (predictands) 
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assembled for a land station or region for a period of 
several years, stratified by season or month. 

The National Weather Service (NWS) has included MOS as 
an integral part of their weather forecasting operations 
Since the mid 1970's and currently forecasts for approximately 
15 weather elements at forecasting times of 6 to 48 hours. 
These MOS forecast equations, developed by the National Oceanic 
and Atmospheric Administration's (NOAA) Techniques Development 
Laboratory (TDL), are based on model output parameters 
(MOP's) from the U.S. regional model, LFM-II. In December 
of 1980, the Air Force Air Weather Service (AWS) also imple- 
mented and operated a MOS forecasting scheme at the Air 
Force Global Weather Center (AFGWC), Offut AFB, Nebraska 
(Best and Pryor, 1983) for approximately 18 months. The 
program was terminated with the decision to replace their 
hemispheric primitive equation model with a spectral global 
dynamic model (Klein, 1981). The linear regression tech- 
niques used by both the Air Force and NWS has demonstrated 
Operationally useful skill in forecasting weather elements 
at locations over land throughout the world (Best and Pryor, 
1983). In this technique, called Regression Estimated Event 
meeeagotiity (REEP), predictor variables are discretized into 
sets of dummy variables prior to regression. 

The Navy's unigue responsibilities of marine forecasting 
provides a motive for it to have its own MOS system. In 


the late 1970's and early 1980's, the Naval Environmental 


iD 


Prediction Research Facility (NEPERF) in Monterey, California 
sponsored a limited amount of research into naval applications 
of MOS, with most of the effort going toward marine visi- 
bility and fog. The results of these Navy studies and the 
encouraging performances of the NWS and AWS MOS programs 
prompted the Navy, in the spring of 1983, to begin develop- 
ment of a MOS program under the guidance of NEPERF re 
forecast operational air/ocean parameters over the oceans of 
the world. The proposed milestones of this ten year projeee 
are summarized in Fig. 1. The first operational weather 
oarameeee investigated in the program is horizontal visi- 
bility over the North Atlantic Ocean using MOP's from the 
Navy Operational Global Atmospheric Prediction System (NOGAPS) , 
a dynamical primitive equation (PE) model run operationally 
at the Fleet Numerical Oceanography Center (FNOC) (Karl, 1984; 
DiUNnt Zo ica 

Previous experimental work by the Navy to forecast 
Open-ocean fog and visibility using linear regression 
equations (Aldinger, 1979; Yavorsky, 1980; =selser7 a 
Koziara et al. 1983; Renard and Thompson, 1984) shows skill 
of marginal operational usefulness but exceeding that of 
persistence and/or climatology. Two factors limit the 
potential for MOS forecasts of visiblity and fog ae "sea, 
First, there is the lack of 'calibrated' fog and visiblity 
observations in that shipboard weather one re lack SUWEifig 


cient reference points to be able to accurately estimate the 


is 





Piiememor AalMOsemermic Visibility. Secondly, the number of 
observed weather reports are not only limited in number, but 
mese come from moving platforms so that there is a lack of 
weather trend information for a single (fixed) station as 

is the case in land observations. Statistical methodologies 
tested by Karl (1984) and Diunizio (1984) to overcome the 
at-sea MOS problems, include a conditional probability 
approach Eropoted by Preisendorfer (1983 a,b,c) and various 
innovative threshold techniques, as applied to the linear 
regression model, developed by Lowe (1984a). 

This study represents a continuation of the North Atlantic 
Ocean MOS studies on visibility, by Karl (1984) and Diunizio 
(1984). However, in this case, the statistical methods 
bested by the earlier visibility studies are applied to cloud 
me@unce and ceiling. The methods used here have been designed 
memoe CONSistent with those of the previous studies in order 


moematlow for comparison of results, as appropriate. 
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Til. OBJECTIVES AND APPROPen 


The objective of this study is to extend the one mee 
NPS research for predicting horizontal Marines 71 cise 
using model output statistics (MOS) (Karl, 1984; Dita 
1984) to the prediction of cloud ceiling) and jeteugeee as 
over coastal and open ocean areas of the North Atlantic 


Ocean. The approach to the problem is as follows: 


A. Define categorical groupings of elloud ameuneeane 
ceiling height which relate to operational use 
ace sear 

B. Determine if one element is important to the 


prediction of the other and, therefore, should 
be investigated first. 


C. Apply the previously investigated methods for 
forecasting visibility to cloud amount and Ceiling, 
and evaluate their performance. These methods 
include Preisendorfer (1983 a,b,c) maximum proba- 
bility and natural regression strategies, and linear 
regression threshold models as proposed by Lowe 
(1984a). 


D. Compare and contrast the results of the two methodolo- 
gies in C, above, and conduct some experimentation 
to improve their applicability to eloud ecoVvereaaa 
ceiling prediction. 


E. Investigate alternative predictor selection schemes 
to improve the ability of the MOS models to distin- 
guish between the predictand categories. 


F. Make recommendations on the usefulness of the 


schemes investigated and potential avenues for 
future work in this area: : 
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mee CLOUD AMOUNT OBSERVATIONS AND SYNOPTIC CODES 

Cloud amount is defined as the fractionof the celestial 
dome covered by all clouds. The observations taken from 
seagoing platforms are reported as values of zero to eight 


oktas (eighths) such that 0 means no clouds, 1 means 1/8th 


mero Cover, etc. In addition, 9 1s used to report an 
@eecured sky (e.g., smoke, fog), for which a defined cloud 
cover 1S not observable. The observations were treated as 


categorized predictands and were divided into categories 
conforming to the standard definitions of opaque sky cover 
for clear, scattered, broken and overcast, as used for 


eemacion observations. 


Cloud Amount Category Eighths Definition 
I 0-4 clear/scattered 
ie Sy broken 
Tan 8 overcast 


The obscured observation was not used in the MOS development 


reported on here. 


[ec EtLING OBSERVATIONS AND SYNOPTIC CODES 
The definition of ceiling is the height ascribed to the 
lowest layer of clouds or obscuring phenomena when it is 


reported as broken, overcast, or obscured and not classified 


i 


as thin or partial. However, the synoptic Surface Marine 

Observations data set does not give a direct observation of 
ceiling height, thus making it necessary for the purpose of 
this study to synthesize ceiling height from the data that 


are given. The data set gives the following reported fields; 


CLAMT : Cloud amount or total sky Jeo 74, 


LOAMT : Total sky cover by low clouds (middle clouds 
if no low clouds are present) 


CLE Height of the lowest clouds irrespective 
Of Same Umer 

Cloud height is reported with a synoptic code from 0 to 9, 
where 0 refers to heights from 0 to 50 feet and 9 refers 
to heights greater than 6500 feet or cases where no clouds 
are present. This 6500 foot height, corresponds roughly to 
the upper boundary of the clouds reported in the low cloud 
amount field, LOAMT, making possible the following definition 
of ceiling for calculational purposes in this study: 

If the reported LOAMT < 5/8 then the ceiling is 

unlimited. 

If the reported LOAMT > 5/8 then Ceilings] ee 

as the reported cloud height. 

The ceiling observations are likewise treated as 
categorized predictands and are divided into the following 


categories for prediction purposes: 
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Ceiling Category Code Definition 


ili 0-3 < 1000 feet 
i 4-5 1000-3500 feet 
aioe 6-9 > 3500 feet 


The above scheme is based on U.S. Navy operational 
eereeria: 
ieee ceiling less than 1000 feet--U.S. Navy aircraft 
carrier at-sea flight recovery operations require 
controlled (IFR) approach guidelines (Department 
Orethce Navy ,. 19 79 jus 


Bee Ceiling 1000-3000 feet--flight recovery operations 
require modified IFR approach guidelines. 


3. Ceiling greater than 3000 feet--at-sea recovery 
Operations change to visual (VFR) approach guidelines. 
C. NORTH ATLANTIC OCEAN DATA 
I. Area 
The North Atlantic Ocean, from 0° to 80°N latitude, 
was divided into homogeneous oceanic areas following Lowe 
(1984b), using a statistical cluster analysis technique. 
The specific homogeneous areas evaluated in this study are 
identified as areas 2 and 4 on Fig. 2. These areas were 
selected because they contain the largest data samples and 
represent two different relative frequencies of cloud cover. 
Area 4 representsS an area where the three categories are of 
near equal population while area 2 represents an area where 
the number of category I (clear and scattered) observations 
1s about one-half of those in categories two and three 


(broken and overcast). 


21 


2. Time Period 

Data from mid-May 1983 to mid-July 1983 were .eonsimes 
to form a more extensive data set, hereafter referred to 
as FATJUNE 1983. FATJUNE 1983 was selected as the initial 
data set for the visibility studies due to its high frequeme: 
of occurrenceof poor visibility observations, and it was 
chosen for this study to maintain continuity on the overall 
MOS project. 1200 GMT synoptic ship report data were used 
exclusively in this study since 1200 GMT corresponds to general 
daylight conditions over the North Atlantic Ocean during 
FATJUNE. For the purpose of this study, TAU-O00 model soucems 
parameters (MOP) generally represent six-hour model forecasts 
valid at 1200 GMT. However, three specific fields, namely 
temperature, geopotential height and wind, are model initiali- 
Zation fields at 1200 GMT. TAU-24 and TAU—43 MOP Ss) ave. 
hour and 48-hour model forecasts, respectively, valid at 1200 
GMT. TAU-00, TAU-24 and TAU-48 MOP's (predictors) are 
employed in the 00-, 24- and 48-h forecast schemes, respectively. 
Summaries of the cloud amount and ceiling frequencies for 
each category type, as a function of homogeneous area and 
prediction time for FATJUNE 1983, are contained im) Tabla 
through IV, respectively. 

3. Synoptic Weather Reports 

All aca weather observations (predictand data) 

for this study were provided by the Naval Oceanography 


Command Detachment (NOCD), Asheville, North Carolina which is 


DZ 





Se Vocated with the National Climatic Data Center (NCDC). 
The observations which were obviously erroneous, as deter- 
Mined from the data quality indicators provided with the 
data, were deleted from the working data sets. 

4. Predictor Parameters 

Pitwegiw nO eiiteey Pour TAU-24 and fifty-four TAU-48 
model output predictors (MOP's) were provided by the Fleet 
Numerical Oceanography Center (FNOC), Monterey, California. 
These parameters are generated by their current operational 
atmospheric prediction model, the Navy Operational Global 
Atmospheric Prediction System (NOGAPS). All MOP's were 
gneerpolated from model grid coordinates to synoptic ship 
report position uSing a linear interpolation scheme. In 
SeeresOn CoO the initial group of model output parameters, 
ten derived parameters representing calculated quantities, 
Such aS parameter gradients, products and advections, were 
included as potential predictors. A listing of all avail- 
able TAU-00, TAU-24 and TAU-48 MOP's are included in 
Appendix D. 

For each homogeneous area and model forecast projec- 
tion, a set of two linear regression equations, in addition 
to the aforementioned MOP's, were included as potential 
MOP's for a separate evaluation of the Preisendorfer 
methodology (the PR+BMD model). These two predictor 
equations were obtained from a standardized linear regression 
software package, P9R, an all possible subsets regression, 
as addressed in the BMDP Statistical Software (University of 
@errornia, 1983). 


Zs 


D. TRAINING/TESDTiNG PAS Se tc 

One-third of the observations were withheld from the 
developmental model to use as an independent data set (the 
testing set). This was accomplished by the use of a counter 
and transfer statement in the computer programs which pre- 
vented every third observation from entering the develop- 
mental computations. Although the approach has the advantage 
of simplicity, there could be some sort of ordering tie 
data base, hence the split runs a chance of being non-random. 
To ensure that the dependent (the training data set) and 
independent (the testing data set) data were representative 
of the same population, a 95% confidence interval for propor- 
tions (Miller and Freund, 1977) was established from the 
entire data set, for each of the weather element categories; 
the training and testing data sets were constrained to have 
frequencies of occurrence within these established confidence 


THeEerVals. 
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ieee 2 ReeeoURES 


Peele RMS AND SYMBOLS 
The terms and statistical symbols defined below will 
be used throughout the remainder of this report. The formal 


mathematical definitions are described in Karl (1984). 


a. 


Maximum probability strategy--choosing the forecast 
weather element (e.g., cloud amount or ceiling) 
category based upon the highest probability of 

the weather element within a predictor interval, 
hence conditional probability. 


a. MAXPROB I--designation of the maximum probability 
strategy in which ties of the highest conditional 
probabilities in a predictor interval are 
resolved by the generation of a random number. 


Db. MAXPROB II--designation of the maximum probability 
strategy in which ties of the highest conditional 
probabilities in a predictor interval are 
resolved by assigning the lowest element category, 
of those tied, as the forecast category. 


Natural regression strategy--choosing weather cate- 
gories based upon the statistical average of the 
conditional probabilities of the weather element 
within a predictor interval. 


AO--the probability of a zero-class weather element 
moeegOryerorecast error (€.g., if cloud amount 
category I is forecast and observed). This is more 
generally known as total percentage correct. 


Al--the probability of a one-class weather element 
category forecast error (e.g., if ceiling category 
I is forecast and category II is observed). 


A2--the probability of a two-class weather element 
category forecast error (e€.g., if ceiling category 
I is forecast and category III is observed). 


CE--class error parameter defined as Al+2A2, used as 


mie pDGEimMary aid im identifying the first predictor 
for the Preisendorfer strategies. 
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ee 


PP--the potential predictability Of tne weaees 
element by any given predictor. Potential predicta- 
bility of a predictand/predictor pair 1s Gepumeame 
Karl (1984) as 


m n 

pp(2|1) = n/(n-1) J) p,(i)t J (PL, 05]ai) - 1yn)7] 
, aE Zak 
i=l J=1 

where: 
P, (i) = the marginal probability of a 
predictor; 
Poy (sia = the conditional probability oneem 


jth predictand, given the wem 
Dpred1 Crem. 


EPI--equally populous interval used to discretize 
the predictors (1.e., subintervals of equal popula- 
tion size based on the predictor range of values). 


Functional dependence--a measure of the stochastic 
dependence of one predictor upon another. Functional 
dependence is the probability that one Of the predmee 
tors will change when the other Changes. Shien eeuaas 
tional dependence values between one already selected 
predictor and another potential predictor indicates 
that little additional information beyond ene weaaa- 
selected predictor is possible. Conversely, a low 
functional dependence value between the same two 
predictors, indicates that each predictor possesses 


distinct information about the predictand. Functional 
dependence range is 0.0 to 1.0 (1.0 = highest func- 
tional dependence). The specific derivation and 


mathematical description of the concept of "functional 
dependence" is discussed in greater depth by 
Preisendorfer (1983c). 


Root-sum-squared functional dependence--the functional 
dependence of a predictor on all predictors already 
included in the developmental model. It is equal 

to the square-root of the sum of the squares of the 
individual functional dependence values. 


TSl1, TS2, TS3--threat score for weather element 


category I, II and III, respectively, computed from 
a contingency table (see Appendix E). 
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3. 


SOMPUTER PROGRAMS 


Four computer programs were developed by Karl (1983) to test 


the proposed Preisendorfer (1983 a,b,c) methodology for forecast- 


ing visibility. These programs were rewritten to allow them to 


be applied to cloud amount and ceiling forecasting and are on 


meee im the Department of Meteorology, Waval Postgraduate School, 


memeerey, California, 93943. 


ie 


A program to compute AO, Al, CE and PP for all predic- 
tors, all strategies (MAXPROB I, MAXPROB II and natural 
regression) for a particular number of equally populous 
predictor intervals. Statistics for the three strate- 
gies are based upon the predictor(s) that proved 
optimal for each strategy. 


a eeogram ro: compute functional dependence for all predic- 
tors, on a given predictor, for a given number of equally 
populous intervals and to compute the associated 96% 
critical confidence interval value (referred to as func- 
tional dependence (96) in this study) by Monte Carlo means. 


PEP cOGdua@meenmcONnStaueGt Contingency tables and to compute 
Skill and threat scores, for both the testing and training 
Gla wa. 


Pweseguanmeowgenerare 100 random data sets, from the 
Marginal probabilities of the predictor(s) in the 
geveloomental model, and to compute upper and lower 

5% Critical confidence interval values for AO and Al 
to be used for testing the significance of the results 
Bem each Of the Preisendorfer models against chance. 
These confidence interval values are calculated via 
Monte Carlo means. This study developed another testing 
standard derived as a consequence of the central limit 
theorem. It is used in the results section to discuss 
Biemealonlricanee Of thetresults of each of the models 
used, and is presented later in this chapter. 


A second set of programs was used to develop the regres- 


sion equations taken mainly from the BMDP STatistical Soft- 


ware Package (University of California, 1983). 


i. 


BMDP P9R. An All Possible Subsets Regression program 
Msaed tO Initially select predictors beginning with 

a general screening of the entire set of potential 
predictors. 


oy 


2. BMDP PIR. A straight regression program) Come saee 
the prediction equation using the variables selected 
by the P9R. 

3. BMDP P5D. This program takes the developed prediction 
equation and produces histograms of the data set 
divided into the prediction categories. 

4. A program to generate the thresholds used with the 
regression equations. (These will be discussed in 
more detail later in this chapter.) 

5. A program was developed to construce coneimngena 
tables of skill and threat scores from the regression 
equation experiments for both the training (dependent) 
and testing (independent) data sets. 

CC. MODELS 
l. Preisendorfer PR Model 

This model represents the first of two different 
applications of the basic Preisendorfer methodology 
(Preisendorfer, 1983 a,b,c). Karl (1984), in his preliminary 
research, provides a rigorous interpretation and results 
associated with this approach. Karl's study provides the 
necessary background for the continuing MOS studies using 
this model. This material will not be repeated here. 

The PR model utilizes the working set of NOGAPS model 
output parameters (MOP's) and derived parameters (Appendix D) 
as potential predictors in constructing a developmental meds 
based upon the training data set, which provides the struc- 
ture by which the testing data set is tested and evaluated. 
In general, these potential predictors Bay si 


values partitioned into discretized equally populous predic- 


tor intervals ("cells"), and conditional prebab li peteonee 
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the predictand are calculated according to the three cate- 
fertes £or cloudsamowunt and ceiling, specified in Chapter 
Iii. Three separate strategies, for determining the specific 
category which is to be identified with each predictor value, 
are proposed. These strategies, two based upon maximum 
probability and the third based on a natural regression 
approach, are addressed as MAXPROB I, MAXPROB II and natural 
regression (NATR) in the remaining portions of the study. 
Initial evaluation of this model involves varying 

meemegualiy populous predictor intervals from sizes of four 
memeug) ten, and selecting an optimal first predictor which 
provides one of the following requirements in the designated 
order: 

Memeo Owest CE value of all the potential predictors; 

Meeecne highest PP valueseofeall the potential predictors. 

Once a first predictor is identified for each of the 

M@ieseirOugh ten equally populous predictor intervals, 
SwieaesDOnding category I, II and III threat and AO skill 
Seeres (Appendix E) are calculated for both the dependent 
and independent data sets. The practice of selecting an 
Optimal equally populous predictor interval (optimal in the 
Pe@eeeOL Maximizing AO) from the eligible grouping sizes of 
four through ten, was proposed by Karl (1984) and used by 
Diunizio (1984) as a practical procedure which would permit 
the realization of peak skill scores as well as maintain 
associated computer storage requirements at a manageable 


level. An unfortunate consequence of this range of potential 
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grouping sizes is that certain statistical Calculatieneeaeees 
Clated with equally populous predictor intervals Ofeemene 
nine and ten are terminated before completion due to a two 
mega-byte storage ceiling at the NPS W.R. Church Computer 
Center (Diunizio, 1984). When considering potential™pre= 
dictor intervals, the size of the interval is of obvious 
importance, with lower values being the most desirable. In 
the previous studies in the MOS series concerning visibility, 
the criterion for determining the optimal equally populous 
predictor interval was to select the smallest interval value 
which maximized the dependent data set AO and independent 
category I threat score. The threat score for category I 
was selected for this purpose because it was felt that low 
Visibility (represented by category I) was uniquely important 
to forecast. In dealing with cloud amount there is not a 
Single category that is obviously most important to fore= 
case, and therefore, the selection of the interval was based 
only on the maximized dependent AQ. This interval was then 
fixed for all ensuing aspects of the model evaluation. 
Consistent with the findings of the previous studies, these 
selection criteria are based on the MAXPROB II scores, 
hence the MAXPROB I and natural regression strategies play 
no role in the predictor selection scheme. 

Once the first predictor and its associated equally 
populous predictor interval have been identified, a functional 
dependent test of the first predictor against the remaining 


potential predictors is run. The second, @avrawanw es 
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subsequent predictors are selected only if both of the 
following criteria are met: 


a. subsequent predictors must increase AQ over the 
AQ value attained at the preceding level, and 


b. the selected predictor must have the lowest root- 
Sum-square functional dependence of all the 
remaining potential predictors. 

Significance tests were run on the developmental 
model after each predictor selection stage had been completed 
to determine if the results were suitably significant as 
compared to random chance. This was accomplished using the 
previously mentioned Monte Carlo method generating the 05 
and 96 percentile confidence intervals using 100 randomly 
generated date sets. Further consideration has brought out 
that 100 cases may not be a large enough sample size for the 
Monte Carlo test. For this reason a testing technique, 
derived as a consequence of the central limit theorem more 
fully described at the end of this chapter, was applied to 
the results at the end of each run to demonstrate that the 
results are significant in relation to chance. 

The model development continues along these criteria 
until computer storage limitations preclude further addition 
of parameters. This generally occurred in previous studies, 
emer every case in this study, at the fifth predictor 
level. Once the developmental model is completed, contingency 
tables of the forecast element category versus the observed 
element category are constructed for both the dependent and 
independent data sets, and threat and skill scores are 


computed and compared. 


onl 


2. Preisendorfer PR+BMD Model 

This model is still the PR model described above, 
but now sets of two linear regression equations are added 
to the list of potential predictors, namely, NOGAPS MOP's 
and derived parameters. 

3. Linear Regression Models 

Linear regression represents the more traditional 
approach to MOS. Regression Estimated Event Probability 
(REEP) is the basis for the National Weather Service and 
Air Weather Service regression models. In this study two 
approaches to the regression model are explored, a single 
stage and a two stage, and three threshold algorithms are 
used: equal-variance, quadratic, and a moditied tana 
likelihood-decision-criteria. The procedures are outlined 
here, but a more detailed explanation of the theories is 
given in Appendix A. 

a. Single Stage Regression 

This model, referred to in the tables as BMD SS, 

consists of generating a single linear regression equation 
trained on the dependent data set, with the predictand set 
equal to l, 2 or 3, corresponding to weather element cate- 
gories I, II or III, respectively. This equation is then 
used with the dependent training set in the graphical plotting 
program BMD P5D,,from the BMDP Statistical Sottwareuee 
generate a set of three histograms and a listing of the 


individual frequency of observation (P), mean (u), and standard 


2 


Geviation (s) of each of the three predictand distributions. 
These statistics are then used in the threshold algorithms 
to calculate two tnreshold values. Finally, the regression 
equation and the two thresholds are used to process the 
independent data to obtain a set of the observed weather 
element versus the forecasted element results in contingency 
table format. These tables and their calculated threat 
scores are presented in Chapter V and Appendix I. 
b. Two-Stage Regression 

This model, referred to in the tables as BMD 
TS, 1s based on a decision-tree scheme using two linear 
regression equations trained on the dependent data. The 
first equation is generated by separating the largest frequency 
Category from the other two. In the cases of cloud amount 
and ceiling this was accomplished by setting the values for 
@eeeooOry I and ii to l and the values for category III to 
2 and then developing a regression equation and threshold 
(as in the single stage above) to suitably describe the two 
distributions. The second stage regression equation and 
threshold are generated, based only on those observations 
which did not exceed the first stage threshold value, effec- 
tively eliminating cases evaluated by the first stage as 
being category III. The second stage is thereby a separation 
Semearegory I from category II observations. In other words, 
the first stage regression separates category III from the 


combined grouping of categories I and II, while the second 
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stage separates the remaining category Il from Caceqemu a 
data. The two resulting equations and associated thresholds 
are then applied to the independnet data to obtain the 
forecast versus observed contingency tables and calculate 
the threat scores. 
c. Threshold Models 

The equal variance model (referred to as EVAR) 
uses an algorithm which requires the assumption that the 
variances of the two normally distributed populations which 
are to be separated by a threshold are equal, while their 
means are unequal. The quadratic threshold (referred to as 
QUAD) algorithm makes no assumptions about the means and 
variances, but does take into consideratiom group epmreas 
probability. The maximum likelihood decision criteria 
(MLDC) was modified for use as a third threshold model in 
order to separate the categories of scattered and broken 
clouds, categories I and II (Cooley, 1978), historically a 
difficult task. The MLDC is not based On apriory Grice 
probabilities but requires only the event conditional proba- 
bility functions of the observations, and is useful in pre- 
dicting events of rare occurrence. In the study, the MLDC 
threshold model consists of using the midpoint between the 
category I and II distribution means with the EVAR thres- 


hold between the category Il and TIf adver eucred. 


D> eo LGM LE LEAN GEES TEhie 
The results of the experiments are tested against two 


standards to demonstrate that the results are significant 
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with respect to chance and to evaluate improvement over 
classical MOS modeling methods. 

Pe omre emcee Ont ilemoxkl ier a Forecast versus Chance 

This first test of thew™results is based on the pro- 

fememmetiiat DOtn percentage correct (AQ) and threat scores 
oly, IS2, TS3) can be presented as probabilities and the 
fact that a binomial population, if large enough, can be 
approximated by a normal distribution. As such the percentage 
Semrect and threat scores may be subjected to a null hypothe- 
Sis Significance test derived as a consequence of the central 
limit theorem. The actual significance testing is made with 
respect to confidence intervals about the scores which would 
be achieved by a uniform random distribution of category 
fee and Ill observations in a 3 x3 contingency table. 
These scores represent the scores which would be achieved by 
pure chance. The test can be stated that the null hypothesis 


is 


with lower limit 


A = y/n - 1.96[y/n(l-y/n) /n] 


Meageeene Upper limit 


B = y/n + 1.96[y/n(l-y/n) /n] 


SD 


A 95% confidence interval is made about the scores expeered 
if each category had an equally likely chance of being fore- 
casted. If the procedure score lies within the interval 
for the null hypothesis score then it is considered that 
there is no statistically significant difference between the 
two scores. The contingency tables and 95% confidence inter-— 
val calculations are shown in Figs. 3, 13, 20, 27)andeee 
2. Improvement Over Baseline 

Since some form of regression is the traditional 
method of developing MOS models, the baseline standard for 
comparison of all the experiments in this study are confi- 
dence intervals generated using the results from the single- 
stage regression for each area, and time period. These 95% 
confidence intervals are made using the same equations as 
the test for significance of a score versus chance. Addi- 
tionally, in area 2, the TAU-00 baseline is used to evaluate 
degradation of the results with time. The baseline intervals 


are shown in Figs. 10, 17, 247530) ance= 


E. MEASURES OF SEPARAR TOT ry 

As the testing proceeded through progressive time stages, 
it became more apparent that the methods were struggling to 
separate the categories of scattered and broken clouds, 
categories I and II (Cooley, 1978). This problem required 
investigation of some alternate predictor selection schemes 
to improve the ability to discriminate between these Ccate= 


gories. Two approaches of determining the optimal separation 
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between the categories were combined and then applied in 
a brief analysis on area 2, TAU-00. These methods are 
termed Class Separability Measures and Cluster Analysis. 
Unfortunately, time did not permit a detailed attempt at 
using these two methods, but the results from area 2, TAU-0O0 
are included in this study and show sufficient potential to 
deserve further study. 
l. Class Separability Measures 

The specific separability measures used were the 
Baettacharya Distance, the Divergence, and the Mahalanobis 
distance (Hand, 1981), each of which is discussed in more 
detail in Appendix B. These measures were calculated using 
the means and variances, and in the case of the Mahalanobis, 
pooled variances of the various predictors with the following 
univariate form: 


Peatcacharya Distance: 


ey 2 2 2 
ie p i caleoa? 
Bh = ( ae) 
52 42 2 Z 01955 
1 2 
Divergence: 
2 2 
Zain l 2) Oo O 
; ii 2 i? ili 
Div a a2 2 + 5 + i = 2 
i 2 Zale oor 
Mahalanobis: 
2 
Mal = 5 
Oo 


The predictors were separated by classes. In this case 
categories I, II and III, and then the three measures were 
calculated for category I versus II, category Tivvesoue 
III, and category I versus III. It is important to note 
that the variables that best discriminate between group I and 
II may not be the same as those that best discriminate 
between II and III or between I and III. In each case the 
means and variances of the predictors were scaled from 0 to 
100 to ease number handling and value comparisons. The 
calculated distance measures are listed in Tables X, XI and 
XII for area 2 at TAU-00" 

2. Cluster Analysis 

Cluster Analysis takes a sample of potential pre- 

dictor variables of unknown classification and groups those 
variables into natural classes or "clusters." The method is 
fundamentally a tool for data exploration to determine if 
natural and useful groupings do, in fact, exist. This method 
was applied to the predictors by use of the BMDP Statistical 
Program, PIM, which provides four measures of similarity for 
clustering variables and three criteria for linking or 
combining clusters. A more detailed discussion of cluster 
analysis is also found in Appendix C. In general, cluster- 
ing was used to determine groupings of predictors that carry 
much the same information in relation to the predictand 


classes. 
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3. Experiments Using Separability and Clustering 
It must be noted that time did not permit an exten- 
Sive investigation of these methods, but rather only a 
cursory look at their potential for usefulness. The basic 
method consists of using the cluster analysis to develop groups 
of variables to choose from, and then employs the separation 
measures to select the "best" predictor from each of these 
clusters. These parameters are then used to develop a linear 
regression equation to predict the three cloud amount cate- 
gories in the same manner as earlier testing in this study. 
Four experiments were attempted: 
a. A Single-stage regression using the variables which 
had relatively high separability measure values 
for category I versus II. 
b. A single-stage regression using the variables 
which had relatively high separability measure 
values for category II versus III. 
c. A single-stage regression using the predictors with 
the highest separation value from each clustered 
SEOup Of predictors. 
d. A two-stage regression uSing separation and 
clustering to separate category I from II and III 
and then category II from III. 
F. GENERAL 
The first area studied was cloud amount in area 4, TAU- 
00, and the procedure is an exact application of the 
methodology used in the previous MOS studies for visibility 
(Karl, 1984; Diunizio, 1984). The one exception is that the 


linear regression model is tested with both a single-stage 


and two-stage regression technique. Next, area 2 of the 
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North Atlantic Ocean was tested at all three time increments, 
TAU-00, TAU-24 and TAU-48, in the same manner, but without 
the two-stage regression. Finally area 2, TAU-00, was tested 
using the measures of separability and clustering techniques. 
Testing on ceiling height prediction was lTimitedmes 
area 2, TAU-00, using initially the same meéthodeleoq =a = 
experiment was then made to test the ability to forecast 
ceilings given perfect skill at predicting clouage cet 
In this case the categorized cloud amount was used jacma 
predictor in the ceiling prediction methodologies aaa. 
results of each of these tests are discussed in the next 
chapter, and are summarized in Tables V through IX and Figss 


4 through 44. 
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Vee ne oULTS 


The procedures for the experimentations on predicting 
cloud amount and ceiling, as specified in Chapter IV, were 
followed for the North Atlantic Ocean homogeneous areas 2 
and 4. These homogeneous areas are displayed in Fig. 2. 

The results of these procedures are summarized in Tables V 
through IX, and detailed results are displayed in Figs. 3 

to 44. This chapter discusses the results and significance 
of each area and each model run using the information on 
these figures. Cloud amount is pursued first in the study 
Since it is important to the prediction of ceilings, as noted 
mieChapter III. 

The terms used throughout this section are defined in 
Chapter IV. The linear regression models are referred to 
as BMD and the three threshold models are Equal Variance 
(EVAR), Quadratic (QUAD) and Maximum Likelihood Decision 
criteria (MLDC). The Preisendorfer method is used both with 
(PR+BMD) and without (PR) linear regression equation predic- 
tors. In each model AO (total percent correct) is used as 
the criterion for the "best" model. In the PR and PR+BMD 
models, a contingency table is generated for all three 
strategies, MAXPROB I, MAXPROB II and natural regression, 
with the addition of each new predictor. In all cases, 


the independent score discussed reflects the best score 
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attained by that particular strategy, while the dependent 
scores reflect the results attained using the first five 


predictors selected. 


A. NORTH ATLANTIC OCEAN AREA 4 CLOUD AMOUNT 

Area 4 was selected as the first for evaluation because 
of its large sample size and nearly equally populous obser- 
vation categories I, II and III. This area encompasses a 
broad region of the North Atlantic Ocean with the southern 
border reaching to the northeastern tip of Portugal and 
extending northward through the English Channel to encompass 
the southern portion of the Noreh Sea “big. 2a 

1. Area 4, TAU-00 (Table V) 

The following are the confidence intervals for 


Significance with respect to chance (Fig. 3): 


Significance Intervals 


AO : 30.53 to Sake 
TS eee meee 
TS2. 20 3o 
TS 3, Loco. Pew 


The first model tested is the two-stage BMD in the 
same manner as the previous NPS visibility studies (Karl, 
1984; Diunizio, 1984). The results, shown In Pigs wads 
5 (EVAR and QUAD), show an AO of 45.27% which 1s Signiticame 
with respect to chance and category II threat score (TS2) 


of .40 which is also highly significant Compared Coueiane—s 
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However, the threat scores for category I and III are below 
mmewconttdencesimterval for pure chance. Th poor results 
are most likely a reflection of the particular nature of 
the frequency of occurrence in each of the observation 
categories. The two-stage regression was chosen in the 
visibility studies because of a very low occurrence (most 
cases less than 5% of total observations) of low visiblity. 
Since low visiblity was the threat most desired to predict, 
the two-stage regression was chosen to more skillfully pre- 
dict a low frequency category. In area 4, on the other hand, 
the frequency of observation is nearly the same for all three 
categories of cloud amount. The thresholds of the two stages 
were moved closer to the middle in the MLDC model (Fig. 6) 
gm Order to better predict the outside two categories, I and 
ieee The resulting AO is 2% lower than the EVAR or QUAD 
models, and an increase in threat scores for both categories 
IT and III occurred. Only the new threat score for category 
I (.33) increased beyond the significance level, but a large 
price was paid in the TS2, which dropped to .27, close to the 
Significance-level boundary. 

since the frequencies of occurrence for the three 
categories are nearly equal, a single-stage regression model 
was next attempted (Figs. 7-9). The EVAR threshold model 
demonstrates only a 1.0% increase in AO, but much more 
importantly, all three categories have threat scores signi- 
ficantly above chance. The QUAD model has a slightly higher 


AO (46.67%) than EVAR and very similar threat scores. Once 
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again, by moving the thresholds with the MLDC model, a 
decrease in AO is observed (43.67%) with significant in- 
creases in TS1l and TS3. "This increase in 7S! andes 
also at the expense of decreasing TS2 below significance 
levels. 

Because the results of the single-stage regression 
were so much better than the two-stage, and in view of the 
fact that all the homogeneous areas of the North Atlantic 
Ocean dispaly similar distributions, the single-stage model 
was pursued for the remainder of the cloud amount experi- 
ments. A single exception will be discussed later. In 
Chapter IV it was mentioned that because linear regression, 
of some form, is the traditional method for MOS studies and 
operational models, it would be selected as a baseline 
measurement (in addition to the confidence interval generated 
by the null hypothesis contingency table) to measure the 
skill of the other methods. The single-stage BMD with the 
EVAR threshold model was selected as this "baseline" measure. 
Fig. 10 shows the development of the confidence intervals 


for area 4 baseline. The resulting intervals are: 


Baseline Intervals 


AO = 43521 tomagae 
TS 122 5 Caen 
TS2:. 9 30) Gomi 
TS34@ 2245 comre 
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The PR model results are shown in Figs. 11-12. The 
model selected a grouping size of six and the peak results 
were attained by all three strategies at the five predictor 
level. Natural regression (NATR) produced the highest AO 
(45.36) for this model, followed by MAXPROB I (44.14) and 
MAXPROB II (41.14). Though all of the strategies had signi- 
ficant AO values compared to chance, none of them improved 
on the AO of the baseline. MAXPROB I improved on BMD for 
TS3, but lagged in other scores. The scores all lie in or 
below the confidence interval for the baseline and, therefore, 
cannot be considered to be significantly different. MAXPROB 
II did appreciably worse in that it showed significance 
with respect to chance but its AO and TS2 were below the 
baseline interval. NATR, with the best AO of the three PR 
strategies, lost skill in category I, as indicated by TSI, 
and this is not even significant with respect to chance. Its 
TS2 and TS3 were not significantly different than the base- 
line values. It is of interest to note that the PR scheme 
and the BMD single-stage model did not select any common 
predictors. 

The PR+BMD scheme selected a grouping size of six and 
attained peak AO values for MAXPROB I and II at two variables 
and for NATR at three variables. In this case both the 
MAXPROB I and II produce near equal results, with MAXPROB 
II showing slightly higher AO, TS1 and TS2. In all three 


Gases the AO was Significant compared to chance but not 
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Significantly different (although lower) than the baseline 
interval. Both MAXPROB I and II improved over their PR 
counterparts in every score except TS3. 

An interesting difference between the Preisendorfer 
models versus the linear regression models that holds 
throughout the study is in the response of the dependent 
scores. In the BMD models the dependent data (training 
set) scone are very near to those of the testing (independent) 
scores, whereas in the Preisendorfer schemes the dependent 
AO scores typically rise to values above 90% with the addi- 
tion of the fifth predictor. This may, indicet= Chama 
PR models do an excellent job of fitting the training sample 
but do not make proper inference concerning the structure of 


the population from which the sample was drawn. 


B. NORTH ATLANTIC OCEAN AREA 2 CLOUD AMOUNT 

Area 2 (Fig. 2) encompasses a geographic region that 
extends from the southeastern tip of Newfoundland, across 
the North Atlantic Ocean to the eastern coast of England, 
north to the Five Fingers of Iceland and back to the Canadian 
coast north of Newfoundland. Area 2 was studied through 
all three time periods, TAU-00, TAU-24 and TAU-48,) anaweaee 
will be discussed separately. As in area 4, a null hypothe- 
Sis is generated for each time period to evaluate the signi- 
ficance of the results versus chance. Also, as in area 4, 
a set of confidence intervals based on the BMD SS model for 


each time period is used as the baseline for measuring 
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PieEOvements accomplished by the other methods in that _time 
Pamloc- tne TAU-O0 baseline 1s used to compare all three 
time frames so that a trend with time can be evaluated as 
well. 
1. Area 2, TAU-00 (Table VI) 
The significance test with respect to chance is 


calculated in Fig. 13 and yields the following intervals: 


Significance Intervals 
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The first model evaluated is the BMD single-stage 
regression using the EVAR threshold, the baseline for 
evaluating other models (Fig. 14). It produced an AO that 
1s significantly better than chance (49.41% compared to 
37.08%) and very significant values for TS2 and TS3. In 
fact, these TS values exceed the EVAR model in area 4, 
TAU-00. However, the price 1s paid in the TSl1 value. The 
Beeeeemode! was able to obtain only a .09 threat score for 
the clear/scattered category, obviously well below the signi- 
ficance test for chance. This is the result of the BMD 
equations being unable to clearly separate the clear/ 
SBeaetered category from the broken category. The historgrams 
Show that the equations result in category I having a mean 


See OS and a standard deviation of .257, while category II 
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has a mean of 2.139 with a standard deviation of 22460Waee 
the means of the two categories being separated by less 

than one-half of a standard deviation, it is easy to see 

why the TSl is so low. Both the EVAR and QUAD models place 
the threshold value separating category I from II well to 

the left of the mean of category I (EVAR threshold = 1.705 
and QUAD threshold = 1.643). This situation holds througheg@e 
the area 2 testing of the BMD model. The QUAD model shows 
only slight variation from EVAR, which is not Sunpremne 
Since the thresholds are very nearly the same. The MLDC 
model moves the threshold between category I and II to 1.864. 
This significantly raises the TSl above the testing confi- 
dence interval, but loses 2% on AO and nearly reduces TS2 
below significance levels. The resulting baseline confidence 


intervals from Fig. 17 are: 


Baseline Interval 
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The PR model, Figs. 18 a-c, selected a grouping size 
of six equally populous intervals and achieved its peak A0 
at the second predictor level with MAXPROB I and MAXPROB II 
(51.26). Both schemes have the same results at this stage, 


not only showing significant results in AUG iSs2eeaeee 
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but also attaining better scores than baseline in all four 

scores. This improvement over baseline however is not 

enough to be above the confidence interval for improvement, 

meenoughn nearly so in both TSl and TS2. NATR did not attain 

feake AQ until the third predictor (48.07) but still 3% 

lower than MAXPROB I or II. NATR displayed an actual signi- 

ficant improvement over the baseline interval in the TS2 

score (.42) but fell below baseline significance in TS3. 

When allowed to progress to five predictors not only did 

NATR continue to improve at the next step, but also each of 

the three schemes improved in TS1l (MAXPROB II scored a 

ime .20 at the fourth predictor) while degrading TS2, 

Moo and AO. This might be significant at some time if TS1l 

were decided to be the most important category to forecast. 
The PR+BMD, Fig. 19 a-c, selected a grouping size 

Of Six, and attained peak AO for MAXPROB I at four predictors, 

and for MAXPROB II and NATR at three predictors.  MAXPROB 

I attained the same AO as it did in the PR model but improves 

[/eoelol and TS3 scores. Although TS1 did not improve to 

Significance with respect to chance, it did improve signi- 

ficantly over the baseline (compare .17 to .11). TS2 was 

nearly equal to baseline, but TS3 was at the upper limit 

Of the baseline confidence interval for improvement. 

MAXPROB II did not fare as well as MAXPROB I overall but TS2 

and TS3 did remain within the confidence interval of the 


baseline, showing no significant difference or improvement. 
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NATR on the other hand attained the highest AO yet received 
by any method or area at 53.43%. This fell .01% above the 
baseline interval and therefore could be considered to be 

a marginally significant improvement over the baseline BMD 
model. NATR also showed significant improvement over base- 
line in TS1 (compare .42 with the upper limiege@ pe ae 
Although NATR remained within the interval of significance 
for the baseline in TSl, it still fell short of statistical 
Significance with respect to chance. 

It is clear by all measures that the PR+BMD method, 
specifically the MAXPROB I strategy, achieved the best 
results. It is significant that none of the methods could 
forecast category I cloud amounts with a skill level better 
than pure chance. 

2. Area 2, TAU-24 (Table VII) 

The TAU-24 time period has an extra five Model Outpue 
Parameters (MOP's) added to the available predictors. All 
other MOP's and derived parameters remained the same (see 
Appendix D). 

The following is the confidence intervals for signi- 


ficance with respect to chance (Fig. 20): 


Significance Test 
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The BMD single-stage regression (Figs. 21-23) starts 
menshiow signs Of deterioration with time as one would expect. 
Again, EVAR has the highest AO at 46.97, which is significant 
with respect to chance and is not significantly different 
from the TAU-00 baseline, but it is near the lower limit of 
that baseline confidence interval. Most of the degradation 
takes place in the TS1 category, which is not doing well at 
TAU-00, but is doing even worse at TAU-24. BMD EVAR and 
QUAD are almost unable to distinguish any category I obser- 
vations from category II (TS1l of .01). At the same time 
both TS2 and TS3 remain within the confidence interval for 
the TAU-00 baseline, showing no significant difference. The 
BMD equation yields an even smaller separation between the 
means of category I and II than was seen in TAU-00. In this 
case, the mean for clear scattered case is 2.068 with a 
standard deviation of .232, and for the broken group, the 
mean is .214 with a standard deviation of .223. The obvious 
problem here is that the separation between the means is 
less than one-third that of the standard deviations! MThis 
is a tough problem for any threshold model. EVAR and QUAD 
produced thresholds of 1.679 and 1.614 respectively, both 
well left of the mean of the scattered cloud group, account- 
ing for the almost zero forecasting of category I cloud 
amount. The BMD EVAR results lead to the following baseline 


mm@eervals (Fig. 24): 


Bel 


Baseline Interval 
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The MLDC moves the threshold to 1.91, resulting in a decrease 
in AO of nearly 5%, falling outside the baseline confidence 
interval. Although the TS1l was raised to .17, 1 ts@nee 
Significant with respect to chance, and the TS2 suffered 
severe degradation such that it is no longer significant 
with respect to chance either. 

The PR model (Fig. 25 a-c) selected eight fon agmaae 
ing size, which limited the model to only four prediceere 
due to a 2 megabyte limitation at the NPS computer center 
(this 1s addressed in Chapter IV of this paper and in Diunizio, 
1984). All three strategies in this time period suffer the 
same inability to forecast category I cloud amount. It is 
not until the fourth predictor that any of the scemes, namely 
NATR attains higher than a .04 TS1. MAXPROB I attains its 
relatively high AO peak (50.31) at the second predictor, but 
is unable to forecast any category I at this level. By the 
fourth predictor it attains a TS1 of .13 while droppings its 
AO to 47.25 and its TS2 (.27) below baseline significance. 
MAXPROB II strongly overpredicts the category Ill oVereae 
Situation, which gives it a TS3 value of (45.5). ities 


is a statistically significant improvement over the baseline, 


SZ 


its TS2 falls below the baseline significance. Its maximum 
AO of 49.49 is not Significantly different than baseline and 
Mmemactained at the first predictor level. By the fourth 
M@eedi ctor it has reached a TS1 that is Significant both to 
chance and the baseline but only with severe degradation to 
Been 1ts AO and TS3 scores. NATR does very poorly overall, 
attaining its maximum AO (45.62) at four predictors, which 
is within baseline interval, but only marginally within the 
TAU-00 baseline confidence interval. Unlike MAXPROB I and 
MAXPROB II, NATR is not able to predict category I with 

any acceptable credibility, even after four predictors. 

ieee too, retained TS2 and TS3 values that are not significantly 
different than the TAU-00 baseline. 

On the other hand, the PR+BMD model (Fig. 26 a-c) 
produced very different results. It selected a grouping 
Size of six equally populous intervals, reaching its peak 
AO for MAXPROB I and II at the second predictor. While 
slightly lower than the AO for PR, the identical results of 
MAXPROB I and II show some skill at forecasting category 
I. MAXPROB I shows a TS1 of .13 and MAXPROB II shows a .l7, 
both of which are significant improvements over the baseline, 
Semi the case of MAXPROB II is marginally significant with 
Beepect to chance. Although still lagging behind in AO by 
nearly 3%, NATR also shows significant improvement over 
baseline in TS1l but not enough to be considered significant 


with respect to chance. 
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In general, it can be observed that an expected 
degradation is experienced from TAU-00 to TAU-24 ieee 
the methodologies. The inability to forecast category Tums 
the most glaring problem. At TAU-00 the skill levels are 
poor in forecasting category I, but in TAU-24 they beeen 
nearly zero in all but the PR+BMD method. In no case are 
any of the methods able to attain significant skill in fore- 
casting scattered clouds in comparison to pure chance. 
However, it would be fair to observe that the AO degradations 
in general are not as significant as one might have expected. 

3. Area 2, TAU-48 (Table VIII) 

The area 2 TAU-48 time period also has the five 
extra predictors mentioned above in TAU-24. The following 
is the confidence intervals for significance of the vse 
scores with respect to chance for TAU-48, area 2 (see Fig: 


Zheyes 


Significance Test 
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As one would expect, the models continue to experience 
a degradation with time. The BMD EVAR model attains only 
an AO of 45.32 which, although well above the significance 
test for chance, still falls below the TAU-00 baseline ¢Giiass 


dence level indicating that it is significantly worse, and 
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that considerable degradation has occurred. Additionally, 
the threat scores for category II and III are only marginally 
within the TAU-00 baseline confidence. Interestingly, 

maeugh Only by a small amount (TS1 = .04), the BMD TAU-48 
[mmoamle tO forecast category I better than TAU-24. The 
Overall degradation in performance is clearly seen in dis- 
Mmemouctons Of the three categories by the BMD equation. 
Ietween the category means for I and II there is now only a 
@ueeicceion Of .05 while the standard deviations are of the 
meer Of .18. This same degradation is seen in the separation 
@emecategories II and III, where the means are now 2.192 and 
2.270 and the standard deviations are .1/8 and .189, respec- 
mevely. This shrinking of the separation of the means is to 
Bi@empolnt at TAU-48 that the QUAD model is unable to produce 
me@en-imaginary threshold between category I and II. MLDC 
also performs consistent with previous time periods, this 
time reducing the AO to 42.93 which is only 6% better than 
fmeemupper Limit on chance. In fact, only the MLDC TS3 

proves to be significantly better than the pure chance 
contingency table. These results lead to the following 


TAU-48 baseline interval (Fig. 30): 


Baseline Interval 
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The PR model (Fig. 3l a-c) also eperiences the same 
degradation with time. The model selected grouping size 
seven and reached peak AO at three predictors for MAXPROB I, 
two predictors for MAXPROB II and four predictors for NATR. 
MAXPROB I loses 3.5% in AO from TAU-24, and also drops below 
the significance limit for baseline TS2. At the same time 
though, it improves on the baseline for TS1, though not enough 
to be considered significant with respect to chance. MAXPROB 
II fares somewhat worse in every category except TS2 where 
it maintains a score within the baseline interval. When com- 
pared for time degradation with the TAU-00 baseline, it is 
only marginally within the baseline interval for AO andeim 
and just below for TS3. Likewise, NATR scores are within 
the 48-h baseline interval, with the exception of TS aoa. 
are significantly worse than the TAU-00 baseline in every 
category with the exception of TS1l. 

The PR+BMD selected a grouping size of six and reached 
its peak AO at the first predictor level. The adGeneve 
scores of MAXPROB I and MAXPROB II show statistically signi- 
cant improvement over the TAU-48 baseline in both AO and 
TS3. However, the TSl1 scores of zero reveal its inability 
at this time period to forecast category I. When the model 
runs out to five predictors, where NATR peaks on AO, then 
it can be seen that all three schemes forecast category I 
with a TSl1 equal to or in excesss of .20. For example, 


MAXPROB II at the fourth predictor has an AO of 45.94, but 
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memotoutricant im alt threat scores with respect to chance 
meee, fot = 726, TS2 = .28, TS3 = .34). NATR again performs 
well below the MAXPROB strategies, even though its AO, 
TS2 and TS3 scores are within the baseline confidence interval. 
In general, it can be said that all the schemes 
suffered significant degradations due to the 48-hour time 
period. Forecasting category I (scattered/clear) remains 
puercolem through all time periods, and is only forecastable 
at large cost to the other threat scores and the total 
memecctitage correct. 


4. Area 2, TAU-00 Experiments in Clustering and 
Separability (Table VII) 


A brief description of the separability and cluster 
methods and procedures is found in Chapter IV, and a more 
detailed theoretical description is found in Appendices B 
and C. The results of the measures of separability program 
are listed in Tables X-XII and the clustering of variables 
is listed in Appendix C. The baseline for comparison in 
these examples is the area 2, TAU-00 BMD using the EVAR 
threshold, and the null hypothesis significance confidence 
intervals used for TAU-00, area, 2. 

The first test consisted of selecting the predictors 
from the category I versus II grouping of the measures of 
separability, using those predictors with the highest 
divergences. As Table X shows, the values for the three 
measures were very low in this grouping, which is a possible 


clue to the low skill attained by all the methods in category 
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I. Predictors were chosen that had a divergence Gieegamem 
higher; predictors were then used in the BMD EVAR model. 
The results of the test are shown in Fig. 33. The model 
attained an AO of 45.39 which is significant with rege awe 
to chance, but is outside the low end of tEheweonm mee 
interval for the baseline. The TS2 is not signitieameie 
different from baseline but TS3 (.34) fell just below the 
lower limit of the baseline value. Most importantly, though, 
the model did not predict any category I. Although this is 
well below the baseline value, the baseline values are 
Significantly worse than chance. 

The second test, found in Fig. 34, is Similar Come 
first, with the exception that the variables were selected 
from the category II versus III grouping of the measures of 
separability (1.e., predictors with values above woe 
These measures showed much higher values, which is consistent 
with the results of the methods in area 2, TAU-00 where the 
threat scores for category II and II are very much higher 


(1.e., the models are able to separate II from III much 


easier). This time the AO improved to 48.91, nearly equaling 
the baseline value. The model also equalled baseline per- 
formance in threat scores TS2 and TS3. Once again, however, 


the model is unable to forecast any category I observations. 
The third test tries to combine the clustering infor- 

mation with the measures of separability. In this case the 

clusters, listed in Appendix C, were used as the initial 


sorting of predictors. Next, the predictor from Gach clusgaem 
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@mae has the highest divergence in the category I versus III 
Mmeouping, was selected as a variable. These variables were 
then used in the BMD EVAR and the results are shown in Fig. 
35. This model did rather poorly and only stayed in the 
confidence interval for the baseline in TS2. All other 
scores dropped below the intervals for baseline while 
femadIning Significant with respect to chance (except for 
et) . 

The fourth test attempted to utilize all the informa- 
tion available. The clustering technique was combined with 
the measures of separability to select variables that would 
best separate category I from II, and then those that would 
Mest separate category II from III. These two sets of pre- 
dictors were then used in a two-stage regression first 
MeMerating category I from II+III and then II from III. 

The results, shown in Fig. 36, show much improvement over 
the previous three tests. In fact, this model produced the 
highest AO attained by any of the BMD models so far studied. 
Mie EVAR threshold produced a 50.59 AO which is higher than 
baseline but not significantly so, and TS2 showed modest 
improvement over the baseline interval. This model also 
produced a smaller TS1 (.05) than hoped for, but the fact 
that it is greater than zero is encouraging. 

It is unfortunate that more time was not available 
to pursue further these methods, but the initial testing shows 
some potential for usefulness in the MOS methods. There are 


several important points to be made. First, the results of 


ye 


the measures of sS@parability program CoOnririmSe Chace 
regression methods used in area 2 are not forecasting category 
I with much skill, because the available predictors do not 
have enough information. Secondly, the cluster mig mee 

to be more valuable if the predictors are scaled some way 

to prevent all the velocity predictors being clustered and 
the height predictors being clustered, etc. (It is posewaam 
that this type of result is not due to scaling but rathereee 
characteristics of the model producing the pavameeoq as 
Thirdly, the measures of separability give high values to 
most of the predictors chosen by the two methods generally 
used in this study. That lends plausiblity to its usefulness 
aS a predictor screening agent to reduce the number of pre- 


dictors being forced through the various prediction strategies. 


C. NORTH ATLANTIC OCEAN AREA 2 CEILINGS 

The first experiments in forecasting ceiling were carried 
out using a direct application of the methods employed for 
cloud amount and previously for visibility. The frequenemes 
of distribution of ceilng observations for the North AtlanGae 
Ocean are shown on Table IV. Area 2 was chosen for experi- 
mentation, consistent with the concentration of MOS visiSaieee 
and cloud amount effort. The second set of experiments is 
designed to evaluate the skill of forecasting ceiling given 
that there exists perfect skill at forecasting cloud amount. 
The cloud amount observations are then categorized and used 


as a parameter in the various methods. As in the studies on 
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cloud amount, a null hypothesis is established, using a 
contingency table, based on each category having an equal 
Pmooability of being forecasted for each observation. This 
meelas the following 95% confidence intervals for evaluating 


the significance of the results (see Fig. 37): 


Significance Intervals 
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1. Area 2, TAU-00 Ceiling Tests Without Cloud Amount 
(Table IX) 


The results of the BMD single-stage Recession 1s shown 
mum. 35. The resulting AO is significant with respect 
Sememance and 1S very Similar to the values obtained in the 
cloud amount studies. Threat scores for category I and II 
are both well above significance with TS2 being the highest 
at 0.42. However, the single-stage model is unable to dis- 
Peemnate between category If and III. This is shown in 
Me@enwtios score of .00 and in the histograms displaying the 
distributions of the BMD equations. The means have good 
MmimeadtlOn between category I and II (1.666 and 1.861 respec- 
mively, with standard deviations of .254 and .225). The 
problem occurs between category II and III where the means 
and standard deviations are 1.852 and 0.192 for category II 


Meee oy Ss and 0.183 for category III. The mean separation 
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is nearly one-tenth that of the standard deviations. iie 
separation is so small that the QUAD model is unable to 
resolve a non-imaginary threshold. In this case the MLDC 
model shows promise. By moving the threshold halfway between 
II and III, TS3 increased from .00 to .15, which is not enemas 
to be significant compared to chance, but it is noteworthy 
that the AO also increased by 1% and there is little effect 
(-.03) on TS2. As in the cloud amount studies, the BMD 
Single-stage regression will be used as the baseline for 
evaluating other methods. In this case, however, the BMD 
with MLDC threshold will be used. This produces the following 


confidence intervals (see Fig. 40): 


Baseline Interval 
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The PR+BMD model chose a grouping size of six and 
attains peak AO for the MAXPROB strategies at the second 
predictor and for NATR at the third predictor (Fig 74a 
MAXPROB I achieves the highest AO (47.24) of the three 
strategies and shows very different results inthe threat 
scores compared to the baseline. It scores well above the 
baseline interval for TS1l and TS3, while showing only 
marginal improvement over chance in TS2. The most striking 


fact is that the PR+BMD does so well in thevecaceQg. 7 aaa 
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(TS3 = .36) compared to the counterpart scores in the BMD 
model. MAXPROB II shows similar results, although slightly 
meee 10 TSl but slightly poorer in AO, TS2, and TS3. 
Actually, MAXPROB II shows no skill compared to chance in 
category II. For NATR, AO peaks at the third predictor and 
attains the second highest percentage correct. In general, 
it does much poorer than the MAXPROB strategies, giving 
results that more closely resemble the baseline results. 
NATR shows no significant difference from baseline in either 
Pemor 152, while scoring Significantly higher in TS3 (though 
only on the margin of being significant with respect to 
chance). The TS1 of 0.18 is not significant compared to 
chance, but it is significantly worse than baseline. 

in general, the methods applied to ceilng heights 
produced very similar results to those attained for cloud 
amount, both in percentage correct and in threat scores. 
BMD with MLDC or even EVAR does the best in forecasting 
category II but is poor in forecasting category I or III. 
Conversely, the MAXPROB strategies are much better at fore- 
casting categories I and III, but at a cost of reducing the 
results for category II below significant levels. 

2. Area 2, TAU-00 Ceiling Using Cloud Amount Observations 

In these experiments cloud amount observations were 
Meee sorized and used as a predictor for ceiling. 

The results of the BMD model using cloud amount 
(Figs. 5, 42-43) are excellent compared to the results 


attained so far in this study. The EVAR model attained an 
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AO of 67.50 which is 18% higher than the baseline BMD. The 
threat scores are consistently high as well, far exceeding 
the baseline in every category by .10 to .44. QUAD provides 
nearly identical results across all categoer rece 

The PR+BMD model is then run making cloud amount 
available as a predictor (Figs. 5, 44 a-c). The model chose 
grouping size six and attains peak AO at the second predic- 
tor. Cloud amount is the first predictor chosen and the 
linear regression equation variable (not containing cloud 
amount) is the second. MAXPROB I and MAXPROB II produce 
identical results at this level with an AO of 68.60, about 
1.0% higher than the BMD model using cloud amoume ae 
model's TS3 is an outstanding .72 and TS2 as.249, bother 
which are very Significant with respect te chance sande 
baseline. However, the resulting TSl is only .24 which is 
Significant compared to chance but is not an improvement over 
the baseline. The NATR model attains peak AO at the fourth 
predictor, and does not perform as well as MAXPROB in any 
category. The TS3 of .64 and the TS2 of .451 show signvGia, 
cance with respect to both chance and the baseline, but the 
TSl value is not significantly different than baseline, 
and only marginally significant compared )remet ues 

The value of good skill in predicting clouGgwaneume 
in forecasting ceiling is obvious by the above results. The 
very high category III results (i.e., ceiling greater than 


3500 feet or unlimited) is probably due to the definition 
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Of ceiling, namely that if cloud amount is less than 5/8 
eno cel ice testlntinteea. The strongest effect of 
cloud amount in the forecast of ceiling is whether or not 
a ceiling exists, thus the high threat score of category 


ITII which contains all the observations of no ceiling. 
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VI. CONCLUSIONS AND RECOM Eo 


AS “CONG@EUS TONS 

The primary objective of this study was” €6 Deqaieed— 
investigation into statistical forecasting of clougeanenme 
and ceiling by extending the methods researched by Karl 
(1984) and applied by Diunizio (1984) in the area of visi- 
bility. The ultimate goal is to develop a viable statisti- 
cal forecasting scheme suitable for eventual employment in 
an operational U.S. Navy marine ceiling and cloud amount 
MOS forecasting system. This is certainly not an exhaustive 
study of the subject, but does provide an important first 
step in statistically forecasting these weather eclementse 

The results of the tests in the various areas and time 
periods show that the methods evaluated are useful in fore- 
casting both cloud amount and ceilings. Although the models 
are not yet producing results as good as one might desire 
for an operational MOS system, they are forecasting signifi- 
cantly better than pure chance, giving them useful skill 
levels. In area 4, TAU-00, the single-stage linear regression 
performed the best, and became the "baseline" from which to 
measure the other methods. In area 2 the model that scored 
consistently highest in all three time periods is the PR+BMD. 
The general problem experienced by all the approaches in 
area 2 is the inability to forecast the scattered/clear condi= 
tion (category I) with any skill. Significant Skea iee 


category is only attained in a very few cases and then only 
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maomgreat Cost tO the threat scores of the other two cate- 
gories. In the initial ceiling studies, PR+tBMD gives the 
best overall results, but the linear regression is able to 
more skillfully forecast the category II. In contrast, when 
a perfect cloud amount forecast is added as a predictor to 
the ceiling models, linear regression gives much better 
results overall, especially in forecasting low ceilings. 

In the previous MOS studies in this series, a low visi- 
bility situation was clearly the most threatening category 
for operational Naval forces and, therefore, was selected 
as the criterion to maximize as well as to evaluate one model 
against another. In cloud amount predictions, there does 
not exist a single category that is clearly more important 
than the other two. In the absence of a better measure, 
absolute percentage correct was utilized. The study does 
reveal a need to develop some evaluation criteria for contin- 
Semey table output for the MOS project in general. This 
would be of great assistance in the developmental stages of 
parameter selection as well as evaluating the overall per- 
formance of a particular model. The two measures used in 
this study to evaluate significance of the results proved 
to be very useful. The previous studies based significance 
testing on a Monte Carlo scheme evaluating a set of 100 
randomly generated data sets to produce upper and lower .05 
Critical values for AO. The significance test used in this 


study, derived as a consequence of the central limit theorem, 
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proved to be much simpler and less time consuming to apply. 
The test gives a good first approximation of the signifticame. 
of any particular model run. The second tool for evaliacume 
the results, the 95% confidence intervals derived from a 
baseline model contingency table, is very useful in CGoOmpaiaaae 
the improvement of each model, and is especially insightful 
in evaluating degradation of results over the 48-hour time 
period. 

It becomes clear after the first few uses of the various 
models, that the linear regression techniques are much more 
easily handled in the developmental stages than the PR or 
PR+BMD models. When placed into an operational MOS system 
the PR models will require several orders of magnitude more 
computer memory storage space than its linear regression 
counterpart. For these reasons, it would seem that if the 
PR methods tested here are to be of viable use operationally, 
they must be able to perform significantly better Change 
linear regression models. 

The results of the ceiling experiment are very encouraging 
indeed. The first conclusion from this set of experiments 
is that the premise early in the study that good skill in 
forecasting cloud amount will be valuable in forecasting 
ceiling heights is correct. The second conclusion 1s tia 
the results support the idea that good skill in statistical 
forecasting of weather elements is more dependent on having 
good predictors and information than on model type. The 


addition of a single (perfect) predictor, cloud amount, 
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caused both the PR and linear regression methods to gain 

over 20% in percentage correct, or stated otherwise, they 
experienced a 40% improvement in their overall percentage 
correct (i.e., increasing AO from 47% to 67%). This leads 
one to believe that the emphasis in further MOS research 
should not be in pursuing new statistical methods, but rather 
in pursuing new combinations of old predictors, and new 
predictors. 

The results of the separability measures and cluster 
analysis individually are not very impressive. The combined 
use, however, of the techniques with a two-stage regression 
give the highest AO for the cloud amount regression schemes, 
and shows some potential as a predictor selection scheme. 
The benefits of the two techniques is that it gives the 
experimenter some control over the parameter selection 
process, in contrast to the "black box" parameter selection 
by the BIMED statistical software package. These methods 
allow the experimenter to adjust the parameter selection 
according to the category desired to select. For example, 
if the third category is the most difficult or most desired 
category for forecasting, then the measures of separability 
can be used to select predictors providing the maximum 
separability between the desired categories. These two 
methods also can provide a screening process for new param- 
eters. With the present models, to evaluate the potential 
of a single new parameter, the entire model must be run again 


from the beginning. The results of the measures of separability 
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point out one more significant fact. The low values of 
separation between category I and II for all the predterom 
in the area 2 cloud amount study very obviously Cotnere— 
with the inability of any of the forecast schemes to skill- 
fully forecast category I. This, too, supports the posvcmem 
that new predictors or new combinations of predictors are 
necessary to improve significantly the results achieved in 
this study. 

It is of interest to note that the most frequently used 
variables by both the Preisendorfer and regression methods 
include vorticity (VOR500, VOR925, and DVRYIDZ) > lowele,_s 
winds (UBLW, U1000), low level vapor pressures (EAIR, E850) 
and products involving vapor pressure at 700 mb (VE700, 


EE O1On 


B. RECOMMENDATIONS 

Based on the observations made in this study and the 
conclusions above, the following recommendations are offered 
to future researchers: 

l. Interpolate the 12 GMT data base to make TAU-0O0, 
TAU-24 and TAU-48 MOP's available as predictors at every 
observation position. 

2. Interpolate 00 GMT MOP's to the 1200 GMT ship position 
to provide l2-nour histery asS.a new preemiaeer: 

3. If the parameters described in 1. and 2. above were 
available, then a time differencing could be done on the 


predictors to give time trend information to the models as 


To 


an additional parameter. For example, if the particular 
parameter at TAU-00, TAU-24 and TAU-48 is given the designa- 
tion Sl, S2 and S3, respectively, then the time differencing 


could be accomplished thus: 


TAU-00 forecast period would use forward difference 


[3xS3 - 4xS2 + $1]/48 


MAU-24 forecast period would use centered difference 


[S3 - 2xS2 + $1]/24 


TAU-48 forecast period would use backward differencing 


[-Sl + 4xS2 - §3]/48 


These new parameters could then be used as predictors in the 
models. 

4. A new predictor also could be developed at each 
time period by doing a spatial difference across the obser- 
vation points to give a representation of advections (l.e., 
thermal, vorticity, moisture). A potential scheme would be 
to use a centered difference at the observation position. 


If the parameter at the observation point was labeled 


Q(1,3), the east/west advection could be represented by 
ey cx ie) ee — eelotitl ,7) = @©(1-1,3))/2L 
where L is the distance between gridpoints. A north/south 


advection could be represented by 


val: 


dQ/dy(i,j) = {O(i, +1) — OUI eye. 


where K is the north/south distance between grid points. 

5. Develop a scaling process for the predictors, peruse 
to using the cluster analysis, that reduces dimensionality 
while maintaining the structure of the predictor's character- 
istics. This may be necessary in view of the widely ranging 
values of the various MOP's. 

6. Use the measures of separability to screen new 
parameters in order to gain insight into their usefulness 
without having to make an entire model run. 

7. Further pursue the measures of separability combined 
with cluster analysis as a parameter selection scheme in 
association with the linear regression models. 

8. Develop a system of general "measures Of CEtee ris 
ness" for the MOS project, specifically for those predictands 
that are categorized and utilize contingency tables. This 
would provide a means for realistic evaluation of the per- 


formance of the various models tested. 
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APPENDIX A 


LINEAR REGRESSION AND THRESHOLD MODELS 


foe GLLNEAR REGRESSION 

The linear regression techniques used in the study were 
first presented by Karl (1984) and extended by Diunizio (1984). 
The least-squares multiple linear regression problem used in 
the study is the BMDP9R, all possible subsets regression 
Gomputer program, found in the BMDP Statistical Software 
Package (University of California, 1983). 

The BMDP9R program employs a "best" possible subset, 
derived independently of variables or variable sequence, 
calculated from the group of potential predictors. Once this 
"best" subset is identified, a linear regression equation is 
fitted to the data, based only upon those selected predictors. 
The "best" possible subset is identified, a linear regression 
equation is fitted to the data, based only upon those selected 
predictors. The "best" possible subset is calculated by a 
Purnville-Wilson algorithm which provides the user with a 
variety of subordinate subsets in addition to the "best" sub- 
set. Three criteria are available to define the "best" 
possible subset as a function of independent variables (pre- 
dictors) and a dependent variable (predictand): the sample 
R, the adjusted R, and Mallow's Cp. The Mallow's Cp criteria 
is used in this study, where "best" is defined as the smallest 


Cp value. 


WE 


Independent variable selection for the BMDP9R program 
begins with a general screening of the entire set of potential 
predictors. Variables which are identified as redundant, 
linear combinations of other variables, with respect to the 
predictand, are deleted from further consideration. The t 
statistics for the coefficients which minimize the Cp yaw 
for each reviewed subset identifies the "best" subset. The 
number of predictors assigned to each subset can be predefined 
and for this study each subset equation was required to have 
S1X DiEpedmerougse 

The role of regression, once appropriate predictor varia- 
bles nave been selected, is simply that of dimension reduction 
(representing a multivariate structure by a univariate proxy 
which constitutes a classificatory or predictive index). 

This proxy takes the form of a polynomial, linear in its 
coefficients, of the components of the multivariate structure. 
The problem now becomes one of determining the form of the 
state conditional distributions (one for "each creme. 
interest; e.g., one, two and three for ceiling categories I, 
II and III, as used in this study). Once an appropriate 

form has been selected, it remains, then, to determine the 
parameters of the class conditional distributions (e.g., 

means and variances) and then apply an appropriate decision 


criterion or threshold model. 
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pee, LHRESHOLDS 


(Lowe, 1984a) 


ie Notation 


= 
a 
rae = 1 | = 
mie = 0 | = 
Peer 
Poror 


Pate —1 nE =0] 
pare —= 0 1 E = 1) 


eee —1/5 


0] 


P(C =0|E =1] 


Pome — 1 nE =0] 
maer—O0 7 = 1) 


Zz = 


an event; this iS an indicator variable which 
when E 1, the threatening event occurs, and 
when E O, the non-threatening event occurs. 


the classification of an unknown event which 
when C = 1, the event is classified as a 
threat, and when C = 0, the event is classi- 
fied as a non-threat. 


Unconditional probability of occurrence of 
enrear. 


unconditional probability of occurrence of 
non-threat. 


of the lst kind (false alarm) er—w iene = 0) 
Of vwene 2nd kind miss) eee ee — eee 


PEO wpe bebility Of an gerrom of the Ist 
Keaetael 


[ie riremeresanllity Or al ecrror Of the 2nd 
SLigtole 


x 


= sscelass conditional probability of misclassi- 
[oe CemcmeOn di ined t 


= class conditional probability of misclassi- 
fying a threat. 


Pate Bette) P p= 0] . 
aoe O) ire eer — Olle 


a value of the predictive index (equivalent 
EOmy above). 


isn weteenc OE ecaltcrive index On the real line. 
Bemea Aichotomous problem, Z2 is divided into two parts: Zs Za 

Cee= 50m 1f Zé a 

Cue o-eiw 1f Z ¢ Za, 


The decision regions are mutually exclusive and exhaustive 


Cre; Zo NZ, = 0 and Z = Zo UZ,)- 
Thresholds = boundary(s) between decision regions. 
io (a eae = class conditional density of z given 
that Eieween 
D Galle =) = class conditional density of wz Gavem 
that £ = 
A(z) = p(z|E=1)/p(z|E=0) = the maximum Pikelegeee 
ratio (i.e., the ratio of class conditijgmmm 
densities). 
Pp, = pilC=1nE=0] vu (C=-0 GE =i es eee 
probability of error. 
2. Minimum Probability GE Error Gr tecomred 
De = probability of an incorrect classifticaprene 
pawn p(C=1/|E=0] plE=0] + ple=0/8=1) sea 
where p[E=1] + p[E=0] = 1. Note that the events E = 1 
and E = 0 are mutually exclusive and exhaustive. The objective 


is to select decision regions (thresholds) so as to minimize Pe: 


jor Se | tS aL] 


eo KG Oia al] 


p[C =O|E =1] 


p(¢ =1|/E=0] 


J pl(z|E=1)dz = the probabilityeen 
Ze 

0 
misclassifying fe= 2 


f plz|E=l1)dz + “{ (pg =a 
Zed, Zeck, 


- |... ptz | Eee 


1 
1- f{ p(2z|BSatee these are 
Zi substituted 
+ into the 
expression 
{ p(z|E=0)dz for Pp. 
Zea, 
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then, 


Pp, = plE=0! joes —O)dz - plE=1)T1l - { ptz|E=1)dz] 


Zeb, Zea, 


and algebraic rearrangement yields, 


fee «CPE =1) - J Coleen Oy — ple =1) p(z|E=1) dz} 
ZeZ 
1 
In order to minimize Pe: Za (the decision region for C = 1) 


will include all those values of z for which the integrand 
in the expression for P. will be negative. The decision regions 


can be symbolically represented as follows: 


eee lr =O) pez |e =0) = pl[E=1) piz|/E=1l)— 0} 


Mmm iz: P(E =0)] p(zjE=0) - p[E=1]) pi(z|E=1) < oO} 


An alternative representation is given by, 


Z Pees 0] p(2)e—0) > pli) p(z\E =1) } 


pee ee 0 ele — > o(2) 8 =~1)/p(zie =0) } 


Likewise, 


Mee «tz: PIE =0)/pl[E=1) < p(z[EB=1)/p(z|E =0)} 


oo 


These statements can be combined to give, 


c=l1 
p(z|E =1)/p(z|E=0) = Az) plE =0]/p([E =1)] 


< 
c=0 


Thresholds are the value(s) of z for which 
A(z) = plE=0O]/pl[E =1] 


This equation can be solved for z either analytically or 
numerically depending on the forms of the density functions. 
3. Threshold Cases 
In order to exemplify the model, the assumption is 
made that the class conditional distributions are Gaussian. 
There are essentially three distinct cases that can arise. 
a. Case I: Equal variances; different means 


(Referred to as the equal variance model (EVAR) 
in the text) 


p(z|B=1) = k exp{(-1/2)(z-y,)*/o*} 


p(zj|E=0) = k exp{(-1/2) (z -u9)7/0*} 


where: 
je = iO eee 
expi (=1)72) (2 =) °/0*} a Pg 
Az) a EEE Eee a 
exp{(-1/2)(z-uy)*/o} Sy Pa 


i 


——— 


' 


where A is the likelihood ratio and Py = p{E=0] and 


= p([E=1]. Thus, the threshold value is 


Py 


z* = (up tu) /2 + 9 In(pp/P,)/ (uy - Uy) 


Donsity 





Giiaesitication index (2) 


The position of the threshold depends on the relative values 
of Py and Po: The threshold moves toward the group with the 
smallest D.- ie P1 = Po the threshold will be the value of 


z where the densities intersect (1.e., where the densities 


are equal). 


b. Case II: Equal means; different variances 
2 = 
ojexpt (-1/2) (z -u,) Jo} a Po 
A(z) = —2WH — 
Der onZ < p 
Tea /O nd og, 1 
with the threshold 
2 





2o-0° Dn 
—_ eee ( y a 


79 


Note that in this situation there are two thresholds. The 
group having the smaller variance will lie between the two 


thresholds. 


Densily 





Classification vind ex xa@.) 


The thresholds shown are typical of a situation where P, < Bo: 
Note that these thresholds lie between the two intersections 
of the densities. If the inequality of prior probabilities 
were reversed, the thresholds would lie outside of the 
region between the two density intersections. Further, note 
that the decision region for the group Nayving enewtee ee 
variance lies between the thresholds. 


c. Case III: General Solution (Referred to as the 
Quadratic Model (QUAD) in the text) 


p(z|E=1) = k/o, exp{(-1/2) (2 -u4)*/04} 


p(z|E=0) = k/oy exp{(-1/2) (2 = uy) “/06} 
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Z Zane. 
uy) - 205954 In (p99)/P 15,9! 


which is recognizable as a quadratic equation in Zz. 


Os tac)” -/ 24 
where: 
a = of - ae 
il 0 
_ oe 
7 ee 2 re 
ee a ee oe to P5917 F 10? 
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Density 


=e 


Classification ingcrane7. 


The remarks given for the figures in cases I and II are also 


applicable here. More often than not, only one of a pair Of 


thresholds 
interest. 


different, 


important. 


4. 


induced by differing variances will be of real 
If the variances of the two groups are radically 


then both members of the threshold pair become 


The-Maximum-Likelihood-of-Detection Criteria 


For this specific model the following background is 


provided: 

event space: 2 mutually 
Tos T) forecast decision 
=O eaat 


do 1S a COrrecewmeoreeace 


dy is a correct forecast 


exclusive populations 


space: 2 possible forecasts 


ine To actually occurs 


is 1 actually occurs 


Problem: select the decision rule d(z) which maps 


the observation space Z into some forecast space 


in some optimal manner. 
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Z may be an observed variable or it may be an 


Wawra be mine ex -Gerilvyeda from a number of variables. 


For this two decision problem, Z is partitioned 


iacOuewOo pares, 2, and Zy- 


0 
CZ. = dy 1f Ze Zo 
antez,) - ds ie 2 Zy 
where Zo n Z4 = 0 and Zo U a = Z 


The maximum-likelihood-of-detection criteria repre- 
sents the simplest decision model. The basic involves select- 
ing the forecast (decision) corresponding to the observation 
(Signal) which is the most likely symptom of the event subse- 


quently observed. Consider the following example: 


problem: diagnose disease A or disease B. 


The observed symptoms occur with probability 0.75 
for A and 0.1 for B. By the maximum-likelihood-of-detection 
criteria (MLDC), diagnose disease A because A is the most 
likely cause of the observed symptoms (if there is no more 
Mioermation). But if we know that A is rare and B 1S common, 
the above decision may not be optimal and MLDC may not be 
appropriate. MLDC requires only that we know the event 
conditional probability density functions of the observations. 


iinaes is: 


oD 


is used 


p(z|m)) and p(z|T,) 


d, if p(z|,) p(z|T) 
decision rule: d(z) = 


d, <5 p(z|7,) < p(z|t) 


In the following development the Gaussian density 


to exemplify the model. 








a Zoe 

p(z|m)) = 1/V275, exp{-l1/2 ( 5 Jas 

aoe Z-2) > 

p(z|t,) - L/V2m0, exp{-1/2 ( = ee 
p(z|t,) 


definition: likelihood ratio CZ 
p(z][%) 


for convention sake we assume @ Pa. 


ilo Ze Li, aan 
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* note the class having the largest variance has a 


bifurcated decision region. 


In the case where the variances are equal, the 


Situation simplifies considerably. 


“1 
gee = Dean) =o 
ZO (Z> - Zo) eo (Zo = Z4) > 0 
=o 
a 
a 2 a 
Zo (Z) ~Zo) - Oo (25 - Zo) < 0 
qo 
qo 
> —< oe 
22 : (Z) + Zo) 
ori 
" (Z) +Z) 7 . 
a 5 7, 
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6 f De 


It is obvious that z* is simply the average of the 
means of the class-conditional distributions and is found 
at the intersections of the two density curves. 

In the foregoing, normal class conditional distribu- 
tions were assumed. This was done because the Gaussian form 
admits of a rather clean analytical solution. However, the 
general concept of the minimum probable error decision 
criteria may be applied to any form of density function. 
Indeed, the density function of one group need not even be 
the same form as that for another group (one might be exponen- 
tial and the other Gaussian). The difficulty with mostenegs 
Gaussian forms is that they seldom admit of closed analytical 
forms and require numerical means in determination of 


Ehresmo lace 


86 


APPENDIX B 


MEASURES OF SEPARABILITY 


As the testing proceeded through progressive time stages 
in the study, it became apparent that the methods were unable 
to separate the categories of scattered and broken clouds, 
categories I and II. This problem required the investigation 
of some alternate predictor selection schemes to improve the 
ability to discriminate between these categories. 

mine decision information for discriminating between two 
categories comes from two sources: the separation of the two 
means and the difference in the variances. The three measures 
considered in this study are the Divergence, the Bhattacharya 
distance and the Mahalanobis distance. These three measures 
attempt to combine both sources of information to come up 
with a single measure of the ability of a predictor to des- 
cribe the separation in the categories of the predictand. 
These measures are applied in the study by stratifying each 
predictor by event (i.e., predictand category), and calcu- 
lating the mean and variance of the stratified predictors. 
Then, for each predictor, the measures of separability are 
calculated for category I versus II, category I versus III, 
and category II versus III. The results are shown in tabular 
femme in Tables ll to 13. 

The Mahalanobis distance considers the variances as equal 


and uses pooled variances of the predictor in the following 
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Un iVarkeate =o vem 


(Ly — U5) 


o¢ 


It can be thought of as a signal-to-noise ratio where seme 
difference in the two means is the desired signal, and the 
noise is the scatter within the whole set (the variance). 

The Divergence does not assume equal variance, and, 
therefore, does not use a pooled variance. It adds toma. 
Signal-to-noise ratio two quotients of the variances adjusted 
by the equal variance value (two). It has the effect of 
combining the signal-to-noise ratio with information contained 
in the variances. The Divergence is used in this study in 


1ts UNivariace form: 


Sa ie io tes 
1 2 rae il 
2, 2 il 
i ae ie 2 


The third measure of separability applied to the data 
set, the Bhattacharyya distance, 1S a special case of the 
Chernoff distance. Although more complicated than the 
Divergence, it also combines the information contained in the 
mean with that found in the variance. The Bhattacharyya is 


used in the study in its univariate form as: 
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The results of all three measures of separability applied 
to the predictors used in the study, are shown in Tables X 
through XII, for homogeneous area 2 at the time period 


TAU-O0. 
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APPEND] 


CLUSTER ANALYSIS 


The object of cluster analysis is to take a Sampveuem 
potential predictor variables of unknown classification and 
group them into natural classes or clusters. The faceyGieae 
there 1s no a priori classification of the sample sugqgqeau: 
that cluster analysis is fundamentally a tool for data 
exploration. That is to say, one wishes to study the data 
to see if natural and useful groupings do, in fact) exer 
It 1s important to note that for any application of the mega 
there are many possible classifications which can be imposed 
on a sample. Therefore, the sort of groupings which emerges 
from an analysis will depend very much on the variables used 
to represent the predictand. The poor choice of variables 
can lead to a clustering which is useless for a particular 
purpose. 

The clustering done for cloud amount uses the BMDP Sao 
tistical Software (University of California, 1983) PIM prGaae 
applied all available Model Output Parameters (MOP's). The 
PIM provides four measures of similarity (association) for 
clustering variables and three criteria for linking On seem. 
bining clusters. Initially, each variable is considered 
a separate cluster; then, the two most similar variables are 
joined to form a cluster. The amalgamating process continues 


in a stepwise fashion (joining variables or clusters of 
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variables) until a single cluster is formed that contains 
emer the variables. 

As used in this study, the measure of similarity is the 
absolute value of the correlation. The similarity measure 
could also be obtained from a measure of the distance, such 
as the angle between two variables (arccosine of the corre- 
tation) or the acute angle corresponding to the arccosine 
of the absolute value of the correlation. 

The linkage rule (the criterion for combining two 
mimesters) can be the minimum distance (or maximum similarity) 
over all pairings of the variables between the two clusters, 
the maximum distance (or minimum similarity), or the average 
distance (or similarity). The average similarity is the 
mieeemmetic average of the similarity using all possible 
pairings of the variables between the two clusters. The 
maximum similarity (minimum distance), single linkage is used 
Semmene MOP's in this study. 

The output of the PIM program for homogeneous ocean area 


2, at time TAU-O0O, is: 


Predictor Clusters 


Cluster Predictors 
i. PiOvitreee soap Do Za,08D700, D500, D400, D300, D250 
2 Lotta sOOne DDE a Tt 700, T300, TE/00 
S. . VORSU0 Ee VORI 25, DVRTDP 
4. Tee LCOO,. £925 
ay DO vtmmere 0, EI25, EPRD, TEI25, 


Cee Cee th 2 5) 


Oi 


iG 


PBLD, STRITK, REG oe ae 


SM eo 

BVLW, V850, VIZSte oer 
V400, V300, V2Z507 Vizeue 
UDVDZ 


UBLW, US50, U9ZS ulate 
U400;, US00>, UZaG 


DRAG, ETRNMT 


og 


V/00, VS0G8 
VE/00, VE7GGs 


U700, UStie 


eed > 
Meeel output time: 


A. 


ATLANTIC OCEAN, 


Model output 
parameter 
D1000 


Do 25 
D850 
D700 
D500 
D400 
D300 
D250 
TAIR 


T1000 


225 
700 
3010 
T400 
3500 
2 5.0 
EAIR 


E1000 


Bo 25 
£850 
E7/00 
E500 
UBLW 


U1LO00 


W925 


APPENDIX D 


i200 GMT VrAu-00)) 


NOGAPS PREDICTOR PARAMETERS AVAILABLE FOR THE NORTH 
ony 69383, EXPERIMENTS 


Entire North Atlantic Ocean and Mediterranean Sea 


Descriptive name of parameter 


1000 mb geopotential height 


OZ mils 
850 mb 
700 mb 
500 mb 
400 mb 
300 mb 
250 mle 


geopotential 
geopotential 
geopotential 
geopotential 
geopotential 
geopotential 


geopotential 


height 
Me rot 
height 
she suelelie 
height 
height 
height 


Surface air temperature 


1000 mb temperature 


2 oem 
700 Mb 
S00n ms 
400 mb 
300 mb 
ZASNO Sie) 


PemMee ra tlade 
emer Uae 
Eemecracture 
temperature 
temperature 


LemMecr ature 


Surface vapor pressure 


1000 mb vapor pressure 


Zi 
850 mb 
700 mb 
500 mb 


vapor pressure 


vapor pressure 


vVaPOr Prcs sure 


vapor pressure 


Boundary layer zonal wind component 


LOO msmzenal wands component 


925 mb zonal wind component 


5 


U850 850 mb zonal wind component 


U700 700 mb zonal wind component 

Us0G 500 mb zonal wind componeae 

U400 400 mb zonal wind component 

U300 300 mb zonal wind component 

W250 250 mb zonal wind component 

VB LW Boundary layer meridional wind 
CONIPOn Cie 

V1000 1000 mb meridional wind component 

Vos 925 mb meridional wind component 

V850 850 mb meridional wind component 

V700 700 mb meridional wind component 

V500 500 mb meridional wind component 

V400 400 mb meridional wind component 

V300 300 mb meridional wind component 

V250 250 mb meridional wind component 

VORS25 9 2 5 embev.eis he aay, 

VOR500 50 0. embss01t ee, 

BS Surface pressure 

SMF Surface moisture flux 

PBLD Planetary boundary-layer depth 

SLREEO Percent Stratus frequenc, 

SaeRenanet Stratus thickness 

Sie: Surface heat flux 

ENTRN Entrainment at top of marine 


boundary-layer 
DRAG Drag coefficient (Cp) 


Derived parameters 


RELH Surface relative humidity 

DVRTDP Vertical gradient ef vor erema, 
(VOR92Z25 = VORSCO) 

EPRD Product of vapem pressuve- 
(E1000xE850) 

DDDP Height thickness (D925-—D250) 7 Ga= 

VT700 Approximation of thermal advecrmeg 
(V700xT700) 
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UDVDZ 


TE/00 


i250 


BD 25 


Approximation of thermal advection 
(U700x (VLO00-V500) 


Product of temperature and vapor 
pressurex(T/00 E700) 


Approximation of thermal advection 
er Z50xVZ50 ) 


Product of temperature and vapor 
pressure (T925xe925) 


Area: Entire North Atlantic Ocean and Mediterranean Sea 


Model output time: 


1200 GMT (TAU-24 and TAU-48) 


Parameters available and derived parameters at TAU-24 and 
TAU-48 are the same as those for TAU-00 with the addition of 
the following five parameters: 


Model output 


EReECIP 


SHWRS 


INSTAB 


mayo 25 
DIV500 


Descriptive name of parameter 
Total amount (mm.) of model precipitation 
in the last six hours 


LOocateamoune sum.) Of model. precipi- 
Boe lwoneassoclated with cumulus 
SCOmvoecetaene tie tie) laste Six hours 


Boundary layer inversion instability 
925 mb Divergence 


500 mb Divergence 
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TO ec 


Abevdh 


It 


mS2 


Il 


ES 


VERIFICATION SCOR: 


FORECAST 


= R+S+T+U0 


percent correc 


one-class error 


Threat score for 


Threat SGoOLre. form 


Threat score for 


APPENDIX E 


OBSERVED 





DEFINITIONS 


+V+Wt+xX+yY+ Zz 


(X+V+T) /Total 


= (U+S+Y+W) /Total 


Caeegemya 


Category, sel 


Category sul 


96 


X/ (R+tU+X+Y+Z) 


V/ (U+V+W+S+Y) 


T/ (R+S+T+W+Z) 


APPENDIX F 
BMDP LINEAR REGRESSION EQUATION PREDICTOR SETS. 
NORTH ATLANTIC OCEAN (PR+BMD) 
These are the derived linear regression equations used 
as additional predictors in the PR+BMD model. The BMD value 


of each equation represents an estimate of the category 


meedictand. 
r. Area 4, TAU-00, Cloud amount 
ei — eles 7/64 + °0-257546E—-0/7xU850 + 0.372xE700 
= 0245957500 —- 0.00837xSTRFO - 9640.3555xVOR500 
+ 28687.457xVOR925 
BMD2 = -0.293341 - 0.257147xTX + 0.0008191xUBLW 
=m oo xt 500 = 3345-31 xVOR500 
Tome Ivor 7> — 0.002537xEPRD 
, Area 2, TAU-00, Cloud amount 
BMDL = 2.05292 - 0.09055xEAIR + 0.19066E-03xUBLW 
eee Jot Oo xX VORD00 + 7474./707xVOR925 
Om s0 5h PRD +.) 0. 76387E—07xXUDVDZ 
eo 2Z = 2.5l0ls —- O228119E-03xU700 + 0.31987xE500 


+ 1.73035xDVRTDP + 0.27946E-04xU700 


pope ere ~/tZ50 +°0.00236xTES25 


Oe, 


. 


Ly. 


Area 


BMD1L 


BMD2 


Area 


BMD1L 


BMD2 


Area 


BMD1L 


BMD2 


2, 


2, 


TAU-24, Cloud amount 


Z 


-98984 - O.LIZLLXEAIR = OS 630G2 ie 
0.O01415xSHF = 3.28037 - Die 


2.27441 xVORS25 + OS0UGSo a) 


~95832 - 0.05608xTAIR - 0.004547> Gee 
0.01D297xSMF +°02512733~2iyeze 


0.8508E-05xU1T000 + O71o27 3.2 oe 


TAU-48, Cloud amount 


i 


.29808 + 0.35/787E-03xPS = 030002. aaa. 
0.014338xV850 + 0.148c858=03-0500 


0.0412xV500 + 020106577 


45617 - 0.5245XEAIR + Opie] Ee 
0.06068xE925 — 0.23832 -06- cringe 


0.0837xSTRIK = 6-19202> Die. 


TAU=00," Ceiling 


2 


-96681 - 0.03478xE850 + 078364 xE-03>3 
0.03513xT925.+ 0.42945 -02. es 


2759.3096xVOR500 — 05632585 -04 


~78741 - 0.15939xCLAMT —- 0.5971E-04~0Ei 
0.1187E-03xV700 —s0.06024) 222- 


2607/6801xVOR500 - 0.405/E-04xVE/00 
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APPENDIX G 


BMDP LINEAR REGRESSION EQUATION PREDICTOR SETS. 
NORTH ATLANTIC FOR REGRESSION MODELS 


These are the derived linear regression equations used 


in the one and two stage regression models. The BMD value 


of each equation represents an estimate of the category 


Pmeeaictand. 


i. 


Area 4, TAU-00, Cloud amount 
a. Two stage regression 
Vie co Ole Orl45506~TX = 0.004051xSHF 


Porm Lo0saxT O00 0. 000206xUBLW 


SmeegolL/745~UL00G +040 7 3322xE850 


eee Oe 0y 4 e027 64xTx — 0,.010425xSHF 


O20 OGiaeURiWes 0. 11457xTiL000 


a 


moe oloplOog + 0 .0792243xEs50 


b. Single stage regression 
ieee ro 5648-8). O25715xTxX + 0.0008191xUBLW 
POO 2 50) moo 4 >, 5 LxXVOR5S00 


ec lee VOR 2 5 —mUrOU a5 7 <P PRD 


Area 2, TAU-00, Cloud amount 
a. Single stage regression 
eee a2 2 re Oa ~ Eh ATR -e0219066E-03xUBLW 
- 5335.9844xVOR500 + 7474.707xVOR925 
Te 0.00505~EPRD + 027638 /E-7xUDVDZ 
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Single stage regression Separation Test l 
Vl = 1.87611 + 0.98222E-04xUBLW - 0.22933E-04-0 Ge 
- 0.66081xE-04xD850 + .5844E-04xD700 


- 0.02909XSHF + 4128 -5iS2" 2 -2- 


Single stage regression Separation Test 2 
Vl = 1.73895 + 0.05728xE@50 +°O0os7 oa 
- 0.04394xE500 + 1.56723xDVRTDP 


+ 0.21519E-04xVE700 —- 01003357 te 


Single stage regression Separation & Cluster 
Vl = 2.56562 - 0.75713E-03xPS - 0.056507 
+ 0.49973E-04xU1000 + 03079722 
+ 0.00931xSTRTTK + 4624.29688xVOR925 


+ 0.20189E-04xVT700 + 0200127 aa 


Two stage regression separation test 


Vl = 1.3457 + 0.43966E-04xUBLW + 0.5055E-05~D Eas 
+ 0.64366E-04xU1000 - O°61831E-03 32 
+ 0.00285xSHF + 2488.08984xVOR925 

V2 = 2.3040 = 0.01869XEca0R ee ae 


0.06215xE500 + 1. 4689S. Bre] 


+ 0.39909e-05xVE700 - 0°52294c-04- TEs 


TIt. Area 2, TAU=24> )eloud sewn 


Give 


Single stage regression 

Vl = 2.98984 - 0.11211xXEAIR + O3ec0G> bas 
- 0.01415xSHF = 8.28037. 0225 
- 2.27441xVOR925 + 0 700GS6 ae» 
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my. Area 2, TAU-48, Cloud Amount 


ae. 


ae 


Single stage regression 
Vine aoe ee OS24daen IR + 0.15573xE500 
+ 0.06068xE925 - 0.2383E-03xSTRFO 


On O So Veo inik  —ommbo sO 9xXDIVIZ5 


Area 2, TAU-00, Ceiling 


Single stage regression--no cloud amount variable 
Vl = 2.56681 - 0.03478xE950 + 0.18836E-03xV700 
- 0.03513xT925 + 0.4294E-03xDRAG 


Seer o~ VORSD00 ==" 0. 63253E-04xVE700 


Single stage regression with cloud amount variable 
ae oe oy ole 0. 15939xCLAMT —- 0:5971E-04xUBLW 
eC wienor Poe <7 7005-30 06054xE925 


- 2607/6801xVORS500 - 0.4057E-04xVE700 
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TABLE [I 


A summary of 1200 GMT cloud amount 
observations, 15 May to 07 July 1983, 
North Atlantic Ocean homogeneous areas 
as shown in Fig. 1: TAU-00 
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Total CAT I CAT PEt 
11428 4022 4485 
(3S) (2399) 

1686 297 675 
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C200 (.40) 

29 324 3.00 
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TABLE II 


A summary of 1200 GMT cloud amount 
Obseavactens, ls May to 07 July 1983, 
North Atlantic Ocean homogeneous areas 
as shown in Fig; 1: TAU-24 
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Total CAT. CAL 1 Cai Tit 
9416 Soc So 16 2422 
(250) G.35)) ace 

1460 281 583 6 
(LD) (.40) es4 1) 

AZZ 290 50) DO 
(ie 20) (79 9) (.41) 
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TABLE \ ian 


A summary of 1200 GMT cloud amount 
observations, 15 May to O07 3gu 3 ee 
North Atlantic Ocean homogeneous areas 
as shown in Fig. 1: TAU-48 


Total CAM AL CAT ie CAT Ia 
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(35) (239)) ( . 2iey 

kG 36 io Ogi 646 
eS), (.41) (339 
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524 3 234 132 
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276 =) Os. 382 89 
(252) ( 38) ( , OF 
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TABLE IV 


A summary of 1200 GMT ceiling 
Soservations, to May to O07 guly 1983, 
North Atlantic Ocean homogeneous area 
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TABLE X 


Listing of the measures of separability, 

by osc telrommmrommecloud amount categories 

I versus II, North Atlantic Ocean homogeneous 
area 2 at time period TAU-00 


VARIABLE BHATTACHARYYA DIVERGENCE MAHALANOBIS 
PS 0.09467 O77 14S3 0.09295 
TX 0.02457 0.02943 0.02429 
aX 0.04045 0.04401 0.04054 
SMF 0.01079 0.02040 0.00920 
UBLW 0.06747 0.08060 0.06381 
VBLW 0.01148 0.03612 GO, 0O7 Te 
D100 0.08318 0.08845 0.09098 
T1OO 0.00768 0.01029 0.00740 
E100 GrOne7 7 0.01749 0 0117 26 
U100 Croan 7 0.08200 0 10 522 
V100 0.00962 0.03601 0.00566 
D850 0.09544 0.09767 0.09403 
E850 0.00064 0.00169 0.00048 
U850 0.03957 0.03961 0.03950 
V850 0.00910 meron 0.00735 
D700 0.09371 0.09400 0.09329 
700 0.04722 0.04876 0.04746 
E700 Comoe 4 GeOfe2 5 7 0 Ollee 7 
U700 0.01787 On On owi 0.01788 
v700 0.00922 0.01080 0.00891 
D500 ove? s 0.07878 0.07881 
T500 0.03095 0.03293 0.03033 
E500 Sr0023 2 0.01756 Ge Omiomes 
U500 0.00822 0.01458 0.00746 
Vv500 Gre ow 0.01308 OOS 
D400 0.06663 0.06669 0.06650 
T400 0.01693 0.02318 Griemhew 
U400 0.00697 Omron O2 0.00568 
V400 0.01696 0.01763 0.01676 
D300 0.05530 0.05561 0.05502 
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BHATTACHARYYA 
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TABLE XI 


Listing of the measures of separability, 
Dio u-atecec, £OL,e loud amount categories 
I versus III, North Atlantic Ocean 
homogeneous area 2 at time period TAU-00 


VARIABLE BHATTACHARYYA DIVERGENCE MAHALANOBIS 
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Figure 33. Contingency table results for the 
area 2, TAU-00, Single-stage regression, 
predictors chosen by highest 
measures of separability for category I 
versus II for cloud amount 
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Figure 34. Contingency table results for the 
area 2, TAU-00, single-stage regression, 
predictors chosen by highest 
measures of separability for category ia 
versus III for cloud amount 
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Figure 38. Contingency table results for the 
area 2, TAU-00, single-stage 
regression, EVAR model for ceiling 
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Figure 39. Contingency table results for the 
area 2, TAU-00, single-stage 
regression, MLDC model for ceiling 
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Figure 42. Contingency table results for the 
area 2, TAU-00, single-stage 
regression, using cloud amount as a 
predictor EVAR model for ceiling 
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Figure 43. Contingency table results for the 
area 2, TAU-00, single-stage 
regression, using cloud amount as a 
predictor QUAD model for ceiling 


Lie 


-ZToROTperzd puoosses oy} 7e pedASTYyoOe OV 
jUSpusedsepuT wnuTxXeuw 34} YITM peRetToosse Setoos 
asouj 07 Spuodsezi09 etTqeq AduebuTtjuoo 
Sip © ai@nberp pearl! 12) Sle ancugie) ines joao) ie 
buTpnTout Reema I dOWdXYW ey TOF 
(Tepow qWNd+ud) OO-NWL “2 Bere AOF SRzTNSoeA 
atqeq Aouebutquoo pue wesrbetp [TTys but{ted “ePpyp SanbT 4 


G-36.u-1'5 ao 









€ G ( 
Cie eS SYOLOIGIYd 4O YIGWAN 
\ e y ‘ Z 
; 5 a ee | eee ee o 
GU cs) a ~ 
G9 | 662 on = 
oe > Or 
vc ESa meee E o= 2 . 
' vae| es = = — 
0989 (%)0V < ‘ . _2 
‘ -e 
De 
VLVGO LN3IGN3d3Q0NI —_ Z D 
a= ae J / o~ 
a ae = 
= ee / S 
gjAay3sgo f in 
“OD 
ay of 
IG Gil a | a 
~ 3 f oO 
C oe OS eet Sica 
fe ae Cy 
Gi anes |) 2 [II SL “QNI-« |o” 
a [t Sle ONies eal 
Si i Sp e ISL “GNI-* [os 
s OY “CNI - a 


QN39S7 ce 





GOI6 -(%)OV 
o0SN “O4IS “LYAG “TWIG “IWW19 SYOLcIG3Nd 


LWUTO+OWG+Yd - SNIWSI - 1GOYdXHW - 2 LONE - OANYL 
GSNC I SHOGL SNTYOOS “SA S¥OLOTUSY¥d 40 Yarn 


VIVO iNnadnsasaad 








-TOR0Tpead puooss su 3e peASTYyoeS OV 
qZUopuedepuT wnuTxeu 9Uu YFTM pazetToosse SortOOS 
aS0U} 0 SpuodsezzO0D atqey ADuabutzUOD 

SU Og otTpoiG e Se JUNOUeT pnoTo 

ButTpntout ‘sat6azez4#s II dOUdXWW 9Y4F TOF 
(Topou GWwd+ud) OO-ONVEL ‘7 eeze AOF SRTNSoA 


otqe3 AouabutzuoS pue wexrberp TITYS but{Tted “qpp oaAnbtg 


Gea fh 9'S tO 





Cc ze l 
ge CSl SuOLo 10344 40 43GHNN 
- 5 
6b: cSl r 
SQ| 66z| Obl ]27 
Ome tSi > 


09 89 (%)OV 


~m ey 
-~ 
Seco 


VivG INJIZON39dgdCFMII 


Q3IANU3S HO 








£°O 


> O 
S3x00S Ly3dHL ONY OW INGONSd3ON I 


So 





+ 6——_.5 a © 
le6 > €Sl | i a ” 
be | 16 | al 7 ee o 
‘o> ws 

© 
fae cS z [i Sd: ONieeten ao 
a i-Sl ONE =a) e 
on a | G1. “ONT =< ec 
bh 'S1 ce Oise | 
ango34 — 





GOL6 -(%)JOV 


riviag ats tone aa — eS eee 
SANOINKHOSL ONiYOSS °SA SHU 





Osa O41S “TaNG “cCHIG “IHU 1d SxOloteose 
LWH10+0W9+4d - ANITISI - [19YdkWW - 2 YIMY - OONLI 


T favo —. 5 


— ots 
iad 
oo yi 


li 1N 


180 


TION EOL) UPANOImMola Je PoACTYOr OV 

qZuepuedsepuTt wnutxew 943 YFTM pezeToosse Sst0oOsS 

Ssouz O12 Spuodsez105 etTqey ASusbuTAUOD 

Sul, *“AoOROTpermd e se AuNOoWwe pnoToS buTpnpout 

‘soT6beqerzqzs uotssezbar [Teanzeu oy} A0F 

(Tepoul CWd+ud) OO-AWL “2 este AOF SQATNSoA 
astqej Aouebutjzuoo pue werbetp TTTXS HuTTtsd “OPP eaanbdt 4 


Cand 4S we 


~- 
~ 


Cc z t 
VG2 56S | SYOLOIGINd 40 YGSWNN 
S ¥ £ é I 0 
a Ee ——: we eee: 
Gp: ¢cSl D 
m « a : 
° . ~ 
eras ST e -— = i, 
-¢ oa 0O - 
ro 





S9EEQ (%)OV 





Sdx090S LYSYHL ON’ O8 LNUONSd 





fee. oi . ae eS 
CSl | O | I | 99¢ [ ; = 5 | = =o i 
O Na 
AL: 281 | s 111 SL “GNI -« [oe 
| Ly ees | elas 3 11 SL ‘ONE -* |” 
ae ee eae [SL ‘ONI-¢* [Le 
cL oe | OY ‘ONT -o |~ 
| els | op; O ic ON3937 Z 
09°68 '(%)0V | ! : i. 
: oasn ‘941S ‘LuAd ‘ZWid “IWY70 SYOLQI09Nd 





IWU70+0KG+Yd - QNITIGO - YOY LUN - @ YIM - OONYL 
GONMDINHOSL GNINOOS “SA SHYOLOIGaYd JO Yasin 


VLVG INAGNadao 








LIST OF REFPERENGES 


Aldinger, W.T., 1979: Experiments on Estimating Open Ocean 
Visibilities Using Model Output Statistics. M.S. Thesis 
(R.J. Renard, advisor), Dept. of Meteorology, Naval 
Postgraduate School, Monterey, CA, 8l pp. 


Best, D.L., and Pryor, S.P., 1983: Air Weather ese: vwree 
Model Output Statistics System Project Report. AFGWC/ 
PR-83/001, United States Air Force Air Weather Service 
(MAC), Air Force Global Weather Central, Otte oe 
NE 9 Spee 


Cooley, Duane S., 1978: The Use of Model Output Seacis ames 
for Predicting Ceiling, Visibility and Cloud Amount, 
NWS Technical Procedures Bulletin No. 234, NOAA, 
U.S. Department of Commerce, p. 22. 


Department of the Navy, 1979: CV NATOPS MANUAL, Office of 
the Chief of Naval Operations, Washington, D.C., 156 pp. 


Diunizio, M., 1984: An Evaluation of Discretized Condieioms 
Probability and Linear Regression Theshold Techniques in 
Model Output Statistics Forecasting of Visibility over 
the North Atlantic Ocean. M.S. Thesis (R.J. Renard, 
advisor), Dept. of Meteorology, Naval Postgraduate 
School, Monterey, CA, 227 jee. 


Glahn, H.R., 1983: MOS Support for Military Locations) Paa@ 
the Techniques Development Laboratory. TDL Office Note 
83-9, National Weather Service, NOAA, U.S. Department of 
Commerce, 10 pp. 


, and Lowry, D.A., 1972: The Use of Model Gutpue 
Statistics (MOS) in Objective Weather Forecasting. 
J. Appl. Meteor., ll, peo. 12? 


Godfrey, R.S. and P.R. Lowe, 1979: An Application of Model 
Output Statistics to Forecasting the Occurrence of the 
Levante Wind. Preprints, Sixth Conference on Probability 
and Statistics in Atmospheric Sciences, Banff, Alta., 
Amer. Meteor. Soc., Boston, MA, pp. 83-86. 


Hand, D.J., 1981: Discrimination and Classification, eo oe 
Wiley and Sons, New York, N.H.; pop. ee 


Karl, M.L., 1984: Experiments in Forecasting Atmospheric 
Marine Horizontal Visibility using Model OUtput Statistis 
with Conditional Probabilities of Discretized Parameters. 
M.S. Thesis (R.J. Renard, advisor), Dept. of Meteorology, 
Naval Postgraduate School, Monterey, CA, 165 pp. 


182 


wire Ved., 2982 Design of a MOS System for the Navy. 
Miia RepOree NEP RE Contrace No. NOO228=-80-C-LV22, 
Intercon Weather Consultants, Inc., Camp Springs, MD. 


Koziara, M.C., R.J. Renard and W.J. Thompson, 1983: Estimating 
Marine Fog Probability Using a Model Output Statistics 
Scheme. Monthly Weather Review, lll, pp. 2333-2340. 


Mewit, H.L., 1980: SOCAL MOS Celling/Visibility Forecast 
Algorithms. Task 3 Final Report: Forecast Algorithm 
Development, and Task IV, Section I: Test of Forecast 
Algorithms Employing Dependent Data. NEPRD Contract No. 
NOO0228-78-C-3289, Ocean Data Systems, Inc., Monterey, 
CA. 


Lowe, P., 1984a: The Use of Decision Theory for Determining 
Thresholds for Categorical Forecasts, unpublished 
manuscript, Naval Environmental Prediction Research 
Bacility, Monterey, CA. 


, 1984b: The Use of Multi-Variate Statistics for 
Defining Homogeneous Atmospheric REgions Over the North 
Atlantic Ocean, unpublished manuscript, Naval Environ- 
mental Prediction Research Facility, Monterey, CA. 


, 1984c: The Use of Adjusted Scores and Significance 
Tests in the Verification of Categorical Forecasts, 
unpublished manuscript, Naval Environmental Prediction 
Research Facility, Monterey, CA. 


ieeabas, George, 1984: Alaskan Ceiling Height, Visibility, 
and Obstructions to Vision Guidance, NWS Technical 
Procedures Bulletin, No. 334, NOAA, U.S. Department of 
Sommerce, p. /. 


fer, I., and Freund, J.E., 1977: Probability and Statis- 
mes Lor@ingimecers, 2nd. Ed., Prentice-Hall, Inc., 
Bag lewood Cliffs, NJ, 528 pp. 


Naval Environmental Prediction Research Facility, Monterey, 
mee, 1982: MOS Forecasts for U.S. Navy and Marine Corps 
CONUS Locations--User's Manual, NAVENPREDRSHFAC Document 
No. /W0513-Um-07, Commander, Naval Oceanography Command, 
Bay St. Louis, MS, 18 pp. 


Memeson, T.S., 1972: Numerical-Statistical Prediction of 
Visibility at Sea. M.S. Thesis (R.J. Renard, advisor), 
PeeeenGr Meteorology, Naval Postgraduate School, 
Monterey, CA, 33 pp. 


Preisendorfer, R.W., 1983a: Proposed Studies of Some Basic 
Marine Atmospheric Visibility Prediction Schemes Using 
Model Output Statistics, unpublished manuscript, Depart- 
Meme OL Meteorology, Naval Postgraduate School, Monterey, 
ee, 28 pp. 


3.3 


, L983b: Maximum-Probability and Natural-Regression 
Prediction Strategies, unpublished manuscript, Deparemers 
of Meteorology, Naval Postgraduate School, Monterey, 
CAPO per 


, 1983c: Tests for Functional Dependence of 
Predictors, unpublished manuscript, Department of 
Meteorology, Naval Postgraduate School, Memneere, yee. 


De. 


, 1984: Update of MAXPROB MOS Prediction Method, 
unpublished manuscript, Department of Meteorology, Naval 
Postgraduate School, Monterey, CA, 3 pp. 


Renard, R.J. and W.T. Thompson, 1984: Estimating VisSibiiaiee 
Over the North Pacific Ocean Using Model Output Statasigaeee 
National Weather Digest, Vol. 9, NO. 2,7) bp .te= ee 


Schramm, W.G., 1966: Analysis and Prediction of Visto 
at Sea. M.S. Thesis (R.J. Renard, advisor), Depe ase 
Meteorology, Naval Postgraduate School, Monterey, CA, 
224 400- 


Selsor, H.D., 1980: Further Experiments Using a Model 
Output Statistics Method in Estimating Open Ocean 
Visibility. M.S. Thesis (R.J. Renard, advisor) 7 peor 
of Meteorology, Naval Postgraduate School, Monterey, 
CA, 121 oe 


University of California, 1983: BMDP Statistical Sorewanrce 
1983 Edition, Department of Biomathematics, Universi, 
of California at Los Angeles, University™of Califor 
Press, 7/26 pp. 





Yavorsky, P.G., 1980: Experiments Concerning Categorical 
Forecasts of Open-Ocean Visibility Using Model Output 
Statistics. M.S. Thesis (R.J. Renard, advisor), Dept. 
of Meteorology, Naval Postgraduate School, Monterey, 
CA 62 ae er 


184 





EO, 


Litho oe torre UTTONy LIST 


No. 


Merense Technical Information Center 
Cameron Station 
Alexandria, VA 22314 


Library, Code 0142 
Naval Postgraduate School 
Monterey, CA 93943 


Meteorology Reference Center, Code 63 
Pawaeement or Meteorology 

Naval Postgraduate School 

Monterey, CA 93943 


Professor Robert J. Renard, Code 63Rd 
Chairman, Department of Meteorology 
Naval Postgraduate School 

Menterey, CA 93943 


Chairman (Code 68Mr) 
Department of Oceanography 
Naval Postgraduate School 
Monterey, CA 93943 


Dr. Rudolph W. Preisendorfer 
NOAA/PMEL/R/E/PM Bin 015700 Bldg. 3 
7600 Snad Point Way, N.E. 

seattle, WA 98115-0070 


Mr. Paul Lowe 

Naval Environmental Prediction 
Research Facility 

Monterey, CA 93940 


Dr. Robert Godfrey 

Naval Environmental Prediction 
Research Facility 

Monterey, CA 93940 


mie. Mark Diunizio 
meo5>5 Charbono Point 
San Diego, CA 92131 


Director 

Naval Oceanography Division 
Naval Observatory 

34th and Massachusetts Avenue NW 
Washington, DC 20390 


ies 


Copies 


ere 


erage 


ES is 


a 


LR 


BSG: 


IES 


Gy 


io. 


20. 


Bic 


Commander 

Naval Oceanography Command 
Ne Tir slacroen 

Bay St. Louis, MS ™aeIs 


Commanding Officer 

Naval Oceanographic Office 
NSTL Station 

Bay St. Louis, MS "22577 


Commanding Officer 
Fleet Numerical Oceanography Center 
Monterey, CA 93940 


Commanding Officer 

Naval Ocean Research and Development 
ACtilvilty 

NSTL STation 

Bay St. Lous, Moe cerc 


Commanding Officer 

Naval Environmental Prediction 
Research Facility 

Monterey, CA 93940 


Chairman, Oceanography Department 
U.S. Naval Academy 
Annapolis, MD 21402 


Chief of Naval Research 
800 N. Quincy Street 
Arlington,m pee 2717 


Office of Naval Research (Code 480) 

Naval Ocean Research and Development 
ACtIyAey, 

NSTE Stagion 

Bay “St.. boulls , Mo weo so 


Commander (Air-370) 
Naval Air Systems Command 
Washington, DeC. E2058 


Chief, Ocean Services Division 

National Oceanic and Atmospheric 
Administration 

8060 Thirteenth Street 

Silver Spring, MD 202m 


Dr. Alan Weinstein 
Leader, Code 422 

Ocean Sciences Division 
Office of Naval Research 
Arlington, VA. #2227 


186 


ee: 


Lae 


24. 


Zz . 


Ze. 


Zl 


Zon 


inedia Michael L. Karl 
USS Peleliu (LHA-5) 
FPO San Francisco, CA 96624 


Lcdr Mike Wooster 
yay ADA 22 
Meuntain View, CA 94043 


Lcedr Kris Elias 

SMC #1542 

Naval Postgraduate School 
Monterey, CA 93943 


Mr. Gil Ross, Met 09 

Meteorological Office 
Bracknell, Berkshire 

fig and 


Chief, Technical Procedures Branch 

Meteorological Services Division 

National Oceanic and Atmospheric 
Administration 

National Weather Service 

eiver Spring, MD 20910 


Chief, Technical Services Division 
United States Air Force 

Air Weather Service (MAC) 

meer Force Global Weather Central 
Merutct AFB, NB 68113 


Lt Steve Fatjo 

SMC #2412 

Naval Postgraduate School 
Monterey, CA 93943 


co? 











i Thesis 
| W862 
ee ail 





Wooster 

An evaluation of 
discretized condition- 
al probability and 
linear regression 
threshold techniques 
in model output sta- 
tistics forecasting 
of cloud amount and 
ceiling over the 
North Atlantic Ocean. 








Ll art iw ys ‘ft EO ic ee *; Le Sa4ana s'ee - oar oar as i 

a rae ae OR 2 Ts ee Oe OF te F a - on 

bed Pot ake ra sy » aed oy 6 of Ea “dnd ee aed ow ee ie a AD Hh bad aes Phi re Ae ie i ; ; thesW862 
a * Le eee ee ee ere ee re ey gts Or Lae aot §£% 2 t (qa *s 

aS Hote) # CS re Se ee wy rier ph are te ey ea) Pee ET (ery w) aoe a, |, 7 Aes 

Pry ae ya We Reh ed RICE LM A Poe ee ee oe 644.2 ee a ae eer Tor 

Peo A hte porte hdped Shey, 


LD eee pel A Pa | Kae ; eA TA ‘e'6 Fe A as oF : yy} a Ce An even rs cece lied coer 
CE ean a ’ : s 


























Led Ss eed Me Pan 
















Perry er . 





















} . gana dla el | mh j A 
Pree Witten ye ee ee ye et oe tal rey my oh £ ar Ae 4 y PON RINT It Il] {|| il | | 
i Ce A; 4 a a6 i | = 
opto et Oe rey On Ty ere ee re et Te i ri a ee PTY 3 2 ler ar fi S | ih] 
puhet- ded ee OP) pe opt eyecare, cw Pore gercrd eA AY Re Pag as a il | il 
Pe eeah itch det teal te Te ee G Eta Mate AT re i! ened Se Ea ee Lew te { 
hha tha b er e e e 2 ‘ f DO ae ee Se ie ee ey HHI 
wide hebebint ta Roh ed Poe ie F Lore te A E erie . eee i wht a 
spe andl Pee le eed am 3 Week Pe i ee ie et | | | : 
deed cel eT Th ie ee se jm Le ioe ee ee ee aes ee hy | 
MO OA O10 E:GAL had bi tod a ep Tre ee f “Eh? hg? Cut jeeas ae 
vy eA Sei S- 1D atte Ee eo eee oe »4 . “4.4.8 23 43 iM ' 
See oe oe DPN eee ae fy aS Oe a wT Or 04449 : 
SELLE See RE irre oie ie Lore gygee hs >. x 
POPE th OSL SR ay SOY OP DRS POE eee EE 
2¢ eine gave ne te} eae ee Ht here igh ee Te Pe ede oe ee et peered ae PP 
ioe po a Pedi pith de te ee Oe ee i i Peat fe Ie eee Oe pee tt i i 4 cnet DUDLEY KNOX LIBRARY 
ey) Pat Pry yr rrr ye rier ye FPO eee ’ VEY ye yie : 
Ne tw eh Pye Oe re ryy yer rir) Le ar por we ew | = 
* rr a “sorely ern . 


‘pith beets Lee eae 
Seer ee ee eer ity 


* 
eRe pete LOO N PEPE TYE SETS 
Mt) 08 OW at ee ana 


A 9-4 cS8 ee i ie) EE eh é Ae: 
A Pe a ire 
OAT ROR F > Owe ee ree ree SS LES gd 
































































































































































































































































































































































































































































3 r) 
s 
Died tlh Rid eo en Wy whe CUS OA OO wg 8 Ps TIAL : vi 
SS I Pe et oe FRI ry oe rrr re ee A : - SP 
BF OA EMO ei Srtink Met ale ng: cee PRED eee. OD i kee eS ee 7 ae it ae ' U r 
im Soap bed penta dk aoak 3 rt + At Cre +e fee eee ee ee yee ead ttiet . . ‘ 
TO Ob AG Mob ate B sgee Se er iri ern 24 oo Retake) FRA Re ee Oe ee oe ee ee ee D O 
Sita Te aS IS aay lt Rt SR Ba CE ee Pee eee eee eee eee BS lle! 5 ‘ 8 
Ah RB 410 MAE BOE K-8 Ceo COT ied ae FG IG 0 bel Biase ee Oe Bt OK KS OW EM ae Patek S.g00'd, g A Ot o Oe ee e “e 
ee he ee oe © el pee SAR ae SO Re Oe ary a hae Oeeal yo, ec a Y Ot age n y y 
are Mt (Yee r Tre eh apr eee ME Sere Ss RP Ue ee oe ee 6 0066 @4.0n reer i Td Pe a 6 a ® 
Esp OES ao EEN ong pct Py Ot 60 4-6 ie ery 4 Poe toe ee ARUN Mee eed. he 216 RB es wht f PP rary Fr * vi 2 | ‘ U D F oar 
he EO at Cir reat) hee os ar Dee ar et are a a” Sl ee cca Pr He ene a e 84 ‘ a ’ F 
Poi ey AS DSS MERE RoR IRR ey eer FO) ei Dees Ms ehigateles 7 ' : ' 
et A eet etek ert eta ee oo ee ed) & 6.0m ais O 2h Tr fete ett Uh Lee SY ek ory F eae ; ae - 7a 3 é * 
Dire adeeb tated 4-7 Sn id i Pe See Oe J A) oP Ses aot PRCT rerio mea yee Selah aE oY ato ee ae eso 4 wy : D p 2 8 
Day a rome ey a ee ee eo tr) ae ae ee ae Che er PES eR rH ey) Le, TAO at evr re Pare arr Ae F ra ri ie ‘ , Fs 
Pe tr ret eae Pepper ae ie t0 Pe rr rt ig eT. 14's O14 RE Si HE ED HE OGG Pee roel Pe a - | Y ‘ U vs 
Ce OP ee Te ee | pepe pee 2 [yey r oe , Aa UL Ora Cs Wt Oe 0 ae.’ eve re eye eee ire eee a we ar] ee t a? 
Sd Cee eee | Cee ey ate) eC ye 3 otf @< ft. Cre re hee APM LAR eM RL Oe i Sea ¢ , ary Py 
2 ee eee et wre ei wie © | yy er Pe Cr Co COUR RRS 448 OS A rh ee os Ore F Pere mer ae - ‘ 
Bo 0 rd: peg Sot Pad apelin bh Le Ce ee eee ke ar Peg Cee | x RC a ae ae yA Y ee Pe ee P) Pte erie aa - 6 *4 ran F 
adil tek tee Ey) oe iY eee § Pe oe PN ore rer vr er ) dee Wr eae Pay sf \o mae Pe er yw we nwa oa ‘ Y 
bold oe Le ee ee Jere or eee ir i ee olay pry rr: aK Prep he ‘ $4 Oh-g a? 7 | aise iat : ‘ a Bie ‘4 
intohedubeice eet ad Reser wre rit) er] AA a Ae a qs VPP Pe yore ee es Pea or ert ae ere a} i ee ee) ° ar 
SRO MM SO se ae ee Tear Pe ey hy Seen iy fee Sree ae Ais te e 64 ‘ a i i p P 
re hora tral me ey te Pat eit Ra ae eRe ae Py Pear oe yt 8 we ee ee rn ee de el | Le a o rf 
Le ee Ce Ty ee ee ae det cahs a. Ce Re er gue Oo ol Bite cette OL e4 e 7 er) a ee Py , 
Out wea Fol ROD CA so ee ps Cer eit te B40 cca Se Jer ae A et ek A coh 3S di hh a ri ake u ea rn a er , 
Re ones on dpe Una eetoh edt pele it aC un Lemon me rO\bae ia ¢ dietieinate ge U. Fi Bape’ y s y 
plaka sant te wae Me’ Gyn se a Pe oy a e3 seShrnne arr ' rw] PS eo 7 ee e a 5 ‘ ry 
aioe A rto@ihetems 2. &, as, aod ter Pees hr aha tedad ror wi held ALES dt ae Le eee Ty rt oe ma Le t ’ i a 
Af Ces eae EP a ANE PE It te ee Sek bopro Or fina en tae ay Aa eS ol bs ae a 
Death fate te Wir) Paeb b pers nod aL oll Me Te ad ey ae he ee eer Peres aera Oe hoe oa ‘ 
Mapa dias Does ap erie oe ee rs ee ry bak = Nate hell toe 1s oe ie aren ec) ey ety Fp Pa 44 ) i a i) ol * 
; Lee rw ae i yr a oe ery xs et Role eae : Q area Pe ae a a 
Weritluind ita ee TEP Le a eat) Mir eee ft fy eee LT ae ty Peboe es acd ae 1 ae oe : na aa C 
aiden iL dad Perro wre Db DCE Bie he EB le a ab te The ee Pe ae a ar rs ; »’ ] F 
a Fla pind > Ste Gal eungoe ant Pen et ee tee RO Een ie re ae es at y ery i gevs cn deat doce Bie AL La) ear” 6 nae es 
#219 Hien! Seer] rears) Pee TD eee te bee ar ae ae a a CY eas Pie Rey ee re rere ee ere a) ae O 
Tak Pb pot hater eee Seat utd et A oraberenicy SET Ne a te aren ore yy roe bis sh dee ee ee ‘ no 
wee Be eet Oe ere ae eo) Cera re ae era sta ew er (eee ore re ie 99+ Seg antic d ‘ ag ep AL 5 
peep Par Shee SO ee re a 2 ee CO ten LIN Re ERE SE ave Peer ad Pee Deane y 
Ce ee eo ee re oe FU Ld te mi ntrd ve LFW Sat ae pd tig e: re cere Be s a de TY fi et 
aed ee at Pride hie Ue WS Soko PL eh Ob ak ae ee Ae Fie MT Te Ae ‘ a8 $ » 
a i ete aaa Sede paces oe ee ees erty ees ee te eine ery yet yi Ceo ie Caner race yap . u ats ¢ ° . 
fata tori a rlget ALE LP PSE TP Arey PT eel ee) ee Dh of 6 6.4 Sb18 6.0 Dd "Ge GAnee.s Eat oe Fae aL a | a 
preps dy Fob DB DAIE CO oh he Ae THY OOD A bet ; eee ee ee ee "4 ae ‘ . : 
he Sele ats Aled st pope EA Sof ar hd pie OX AOt Ot cg me mas aS TO aa 4 ty i a ie 2 2 F . 
wie tol cera ahs Ag Tae Pa Ly ararea ¢. eg Th ee Ten Tey Roo ir pr ae bm uJ . ery 1 o ® 
ee See ae an stent aL Poh a ek eee rev oT sy RT Ae Oo Get e e a a eaes ot gd F = Li a se ne 
r 3 Fee Diabadegl oc wonvoeand PEL PALE mod ea re es Pe Pe eer . 4 4 ‘ 
Fe eer eerste praia Wb wey Fo ae UPL Se ar ie eres Be eee os a O Oa 
Mehul Pe eee re a Le te ee re CO Ver ' aut so r - 
Palas aed ae ye é Stitt ber wbity 4 ee . eres a eee ee ey a rene ky eee § : : D 
Se ae aes wa ore Od wie " c arr oe Pr 0 7m b33 kt ot whe bye Meets) a an 
PEE ray ee) op aad tt OP Atete teh CM gtd lgie neg # S-temaees ‘ * Oar ri 
th nl ee vd Cee ee pete Pe Toe bh Hes pero eRe le a @eeaqe ae M 
7 mead ee belt Ee eer aries ta re oe eee a Mt SE RY te RY Ot Rr) ae ° te S 
epee NF fii $0 0) get Fhe Sar en ee Prey ert) Sana Cea aT we ae ee ree A i ee ar 4 Pt Pi 
Creatine Bh 5 8 ch ohh Retain ree ab" rreys Ra dsadet tt Marche e if be fb et.s0. 04 6 stent Coc bts ey ae wy A P 
plo he Rt nD tk taser aed Teoh Pre ON Samra ok Wy ne ae PS Par et ee arr ee ne fa ee or , 5 
ont Ts ree ad fh Cl iets PEST te elle ial mt geese Le ee A Oe ee eee ed Pt ae LE gy AEA a “6 4 
Creat ee ae oe rar 6 Ahern Or. De id 1 eat oe ny Ores | revy Ce eae f of ar} 5 
ae oe ee? Te ad fot oy Eat aed alg ord a ee a Pay A o - Pea 
PRT inde Pati tog . pier, Paine Je) cd . 
416 Pale fi Uh abten cee > aed ‘ 
Re ed Gay ere T) ed Ce ee Lo oe a | 1 a) FI A 
a bel et phil vo tise Ceti oe ‘iow 
Busca rh F Lo ree Sey oy "at r A ' 

J BP tadae Ried 18 eae Ore af ee ae . Par) 
wieder a mtbr i ed py ae a ae ic wae oa Oe . - 
wae See ore are a Fy es ae 

re reo piety SS te 7 a. saa" 4 ; i : o> D a 
1 Kd) Mt of Pi bie py O18 Td oA dig or ce Tiel. o roar} @ as (tec a 7 
. Fe Cee Loree 2 noe Birt Dist SS he a Cat a) ay y , S t ur 
ake as oS id rics v0 33 RO eee cL ee 4 Pp p 
= EF deste ‘Ah Ae, = ee, a Ck oP aa 028 ] ‘ 
oe PD TP SS eS a OP rE a i eee r 
fan phan bau ba PS ae Pn ie Doerr fe . 
ve ie tee Sy leh de at ea eh a etry rae se a Por) tea go "@ O° is . oe 
sy pent gh Ae be oie Seer i wiede eh coe Pas CP Pa A 
et eee ay er eee Py) ee ae eeei cihte os ey ee) aes % 
ca) “ rita beds SWAT A Doe hie aad Pee Ce ie ace) 1 tei aly 2 iets. 5 A e ¢ 
a bate ae od et OP es te Caer yt ee pare: oF ath yi ts 
tee a Ou Ri eotee a 1 2 o 
a a esate toe BN. Ave F rd 
eee Lee 68 ‘Se ye margin! a od we 5 
Pari Adee ey The ee r 
de _ eit ae ai ded be " 
’ : ‘4 gig Ate 
ee ae Are of cee Fae Nera to et . 
Dad ace Le anes FIP ied ea Teese B 
ae rrr On PS 2) Pa wre es Vines 
wwe Be ee fia Ar a oF eat A a 
7 Se PS 124 Bae eee Py he ae a te) ny 
ree oye RAIA A ied este 
se red Ue + fr ee p 
et ere eas as ar re Pa - 
Sona ye oY Pi toe a: PP ‘ 
CE Te Cee Leto P ag er : 
eave lw £4 Wessie ae r) rs 
Cv red are ark to Atos Pi wit ° 
Sia ee a SSO, 
cee BE? a A 
FY et oY ae a ot A 2 
Te are 01 Fees CS - 
Ph ed Fe bee s , 
7) a's ‘s AF i. 2 ss 
: Wife % A ie aet 
oe cy r ' 
J 
td ’ 
Wer 
See 
a ey aia ae vs = 
ae pr 5 eae is Oe Le | 
F EL x eid secines 
& oa TALI saan S 
a Rahs as aa iv P PAN eee - 
; a Yer ee A Ss ri 
frrtye- stay! Ye i Hp ne SP Tae ee Pts 3 Por ec 
Oa” has Nott Pale hole ch eet a ta my teary, PIP Rtg SPR BAN - 
So bd bh op ea oo tw FF a te raps Fs US Cs Tey Sets fa H 
Pa BA ee Bee hy GS 7 ty A = 
Bath ee ae Ot ha one $3 bE nae ee eh hh een , ' - 
te EO Sel et ey parr ets fa td ote eee nee cere ey ae s 
bel’ I Bbada ben we od Le eS «Pie Ex: Pri die: aa fal, PT - * 
Reta tn iy Matte ACh! ae pie Oa } Free ak NP e 47 RU ot) ; 
id yore oe tater ia Hepat de Pee eek ar te Ts Pile] ome yCiet | 
Te aida ee POPPE YIP aS RFE. Foye dyes By Ng . 
adel tt 2 “4 Sabai a ny hig tee eae CREAT ar . 
tcp hae hed pees fae Lge 3 Se 
alt a> Ws ae bake) cae bit Ted a) we ‘ x oe hee a ee 
th ee Rae es PE Bade ere oe A ¥ woe ' 
a hata bo Po eT i ere ore bea . us OE by no 
St Rca Sag eet a aL Te ooh 
ot by el ee, pee a2 0 ae et + ¢ Seay Utaines +s n 
ant pebheeed a cir fae had ete St at LM ae PTY © 
bt ein rd jae ale A a he ie tier ry Ee ‘sy, 5 aC x 
tule boats ete eater a ma yo errr TET Ee RIT To ee toe #8 
auld Ni dite folate tink, ‘a Eh bh. a thalberah te ch ote lath TE td he aC, Va kee} ee eee. vrs ws a . aor 
areca aa b a Bea oe Rta cin ba taialet Site Sete ase a eS beh) is Wee RAN. ECan hc eae a es nea oO 
aalesiiedls We Diet RETRY bere eS obig | ic bite wood Bak a RTE SOE Se PC aren Sh SS Dame = 
De New erypetn & ede bys Lae Sa he ee tet ENS OFerasy e815 WN WS tp Pie ic Pace Ai pe eho “v0 a 
beim lk = ba Pes) UMS 0 4 Sot gies Sar a4, bo Ce SEP aes oe a a oe Te a CS jenve Sas ¢ i ares (is = 
I See day etek ier Syne Ss bad Pate ek de tb, ea Prey ene & # mer": Pi +t bo ee LY or pL Re dt Pe antelle b r 
4 / ph we eh ors ae | Dears oe sans, Bes 4 a Si Anes oon oh Lig Bo ‘ 4 a ee 
sith Nate diieae te Sadie Nolet dtaedllle'tt ¢ amie. aT 8 Fee eae AAS PL eee a MET Ley eo gee 5 
teeth tals Lid i tae) Sy on boobed Vara me ePr oa hietd A OT POL, rs eae oe 
eek Bd Se ee ha ee ere dig tn tay rr os ee =~ fv are > qe bie Pas e's - ba eo fe A eh oS mp on 
iene rele 9. Sr Reid de be Bet det bth Bhed bh ih re - wa Se oY wen eX Ya pyeow 9% wee wer 
be 2 die beta nad pablie te Sey hdd hea’ a ee eres Lay eee We peld yA ete er awe Sphie, ote we » 
ar Baha pot he cS toe en Seaeieqinface BS es 5 Her ope Vy td Ce eh Le ee sa) : 
cheated nei? akties , eV ee ey ll a es oe RT rk Par s ve x Sire ear 
ph deerme hada Lek ete 5 a SA er ar aL A eT et Bs ty Nhe ies oo 
eth Bette eat De J a ngs a oh ah Tatoos nar Ae ot Fon a Ly me « ai @ ee Be whe rare,’ : rt sitet 
alata sth tar be hte waitpid PS omeet tho Wecbls Want d eM On. ieee TL . 
pth dn tn eh Ant Aaa wresyae ys: a, ssa ‘5.4 ee elie 2 he on | 5d Pri F 
bth Seer Pe Lordindaedk dete aertrt bees hell Sia party tambo k Ajtai te adh ae ee ae nee G tf s*y # 5 Pr yee 5 
SEY ho eet erene he thea hs aa Rey ek ee or Pay 3 a a $e ee 1 Le BO aa ot Le ee a et ee oe ares fics SR 
paee ts 7th) EA Yomi Bu ay hs UV WIG UN N  Y t ae a Mi asi D. fab Be et Pe | h ay a: er : , ‘ 
led CEN ENN g ~ ie SW ae ERY Sree at i § “ES Cet ot Sth See tt TL ey | a. °< aS hs "%. 8 % n 
pPBELE Lee rdeae 9 Su te i) he Srtnte bere Nw ew! tA SCLC cr ares A oa ‘ be ere cn a | a ’ P 
Lt te SSS Mh oe Th 2 ie La bh ie ier CO ee nS .ot the eS yeeG @ Urey, sae Sit eae ior See, ee bs Sa tA GeO ak MS A A SCC a) a ee ee a aaa eeak errs 
att Le lala thee th Niet tn Ee OS Pi pha LN a tote ee Pale ya) S Lr coker atta hata 4 OE Siac aca ae ag 
dad Ma) itt adetitahet Uh he Th Get is Yo Weck dat 0 Yn © PR sew EN grew a Ge le gs Aad of pe ae » be , ao te De ECL eR Na cy Ce eer 
palace et RCL eer ie bates in PRA Ran Se OTL, LNG We hei? GLEE Gee sen} CR ate A ee 
ee ee oe Lyle” Bo Se he I a aa ve 'ae SO a qn ren Bie y Oy a a ae ee Oe em SLR a fie. * ead ih 
epee Sy fhe: Ber § ¥ Oe OryjteOlee See vie Ae ae ee et oe %S yes as ay 
i s 












BaD ae ay 
A Ie as 













4 Yale ety an) ae. 
“oe 






Wiehe veyhieae Pe ea er Mere tena SDE A tn 
Sy ae on rer re Mecha Barth a, oO ee De et Lt de Eh Ps SM de Oe | ic Oe he RRA a Py’ 
r more RADA ehh era g We bea te 0 AK gig: 0,008 Lee 


a ee a 
Ok ae ae) pa a ee 


SIA ie Te ie de Mlk hear 












OM hd de di I. “A OPS OPe oD Ue, 





Cie ee 
Lh lk Ae eter ho ite RCS Ee TPS te Ce ae uy) ode 


1 

are L salah! vo ee or ee “wee et th ae bata aettte Sh t eald ont ea) ‘ s 

ete Ro iF naa ont ah oie Reale Ce ik het me | ape a nL re Se sy, * i a a) “ om re ew e pages oh RY 
deh be ek a kD Peat oe Ideal Act hoe! eo, beta Pa dae be aD a el a te] ‘5 Ce 
./* yaw Le Deh deed bh Mathie Tp a BR KP he by abe has UP We dO a Be 




















ee cy © % eer zr 
p ve ~ 
Py =f q a rion aed 






























cn bar ROU Ps ge ee a i i es 
pret rag ry: pct OP Fo ra yh ee vee “ee we yt p Ca a ra Abe 
Tepe arg Raa Ue Pah pet wane y ype) piaccartane Ne Wo ye F ah en AL 
ele Seah tore ee LS a tet be Co basta eed Lae The ek AC . 
he tel ee ee Redd pag: set Paaites del deadbeat a Ok ek BT whet, 

he deh hth Let ln al Dat Poe at oe Deck Male tdi alee Metatkok nds Mae Ce Ne 
le be ee os aS pe a nae 





bo ah Pk dah ae hs remns OCU Mae EN 
« 7k hou taal CR ¥e! Phere serra 6 ae wiht y, 
on) a a pala MAgedeth) ag Ae © 08 
Re CU ary bh Me oe] La 
aw @ ysc snes A ELIE kt eS oa 
e Aad swrn cars etqier oad Wye whey fe: nereie 
Ses Te Be pans 


Misia Deptt th fon to Pe bh ie hd On 
Wiel Stn bd daoke bo dol tee) A 

Meehan i Se Peete Mpidedin dr kd ae 
Sh rh eae hh ae | 
Sey ee UE 
libel! ae be li Bh 2 





kar) 
rE) dale Yeo A a a 
te Se he 
A a tae a ae) OL 

ed CI 





ge Sart 
Fe oS ay ey ei 






































































































































** - oa D ts . Pan} 
wa sure 4% a uN'd’e ee © atwie.e ye ot. F are. het RO Lana eC UR Wma ie er cee CS ion 
yah te Sabon tel Sethi Ahm Bate SPC ULL hte wo C RRO RE orth le Arta Beet te et MUN aS ie, Rane ee , ar Cee ine - 
sth ble it 2 Oba ae antes Z + ade ee ee ee yO Seat Ve WIN do Bet, . Yate tl’ oY i bs a Ms . 
Pris dae Bee Pe I eh Sates et ae oy Rie wetye LETT OR Lis ea ee a yi VeBiet ge % a] mea ie a) TP eh Par] ' Py 
sstiedhinda> puted te th eal Tori ne POO CrP hs wey see sh ¢ aw era rer eh Sere « SN aA Oe i Celgene ati +4 ny ss 
tts he | bite da thetee Medd beh e | aay winvete ya Pir eek ei tor ron Tbs a e*hboads ar aS Ve oage 40 5 ’ ae A 
ete Matt hte ee A a ee a Pe ay Ce CHR pace a rar ae? *< ¢ oi ° A 
© Be a A we oh i fe oe en Ps aue Para fe Nar 4 batch ea wi a a “ U a+ ‘ ‘ Uy Can ’ . 
Ria be deat ST | “wow bei, PA StS RCN ER er pc Le ee bes eoErves Cae - 
sere eteissonctye Ca ena : OBO OLS wae - white’ SC ee ee ee Cn ee oe | ache ow 4 . 8 
Hie Al a Le , UT is oe Ye Z 7. \ aon At CS ea « 6 + 4 FS 
et leiet ent icth a eR be o% ORs ee Ok et i a! Lar Of oe i treet + a g ; 
QFUr Swing td Pm gre are eee Ae eae Sa wy Ae * ¢ be tr ASF sc ys a ri 
Pein te Bde | eet Saray Led Ee Bn Dee! el Re a | Ss b) 
COE gg Oy oye Ys ’ Cae aa! ALA Se G2 ie We oe OL ee. x ee eT) = ead, oc SS CT a ere 
ea atb ih Lt) t Shem al, Pe Cre oS B YEE GTO Le. i i a. C.F RRO de. Haag ce 7 4 a 
SORRY PgR aed ene! PR ef Ce ey Cie eyes F as | ay ; cf & ee? Py ‘i a) ry 
Pert cy Ci Comat TAL rr ae bpermm > wre, Ros es = ue Pr 2a Yt eta es he i A hes ee JS BCT Toes. an i ar) ‘ ' 
erate: et Pe ea | iaivon an) 5 "i Oh ae Or eT wrt Sar to's e gee 1 ae ar aan} 7 4 | ee oe? | 8° t 
5 A le Toe et oe pent MPRA Ca A OO Yves es cee te ety ets | fat aS! sae 5 ee U 
bike RE SR Oot aa ’ ) ag OE Ad tke POCO ey Bach Str a ‘we Ae ty ae 
bie he'd Ue STA ee BO Pc LT) pie Se i ae ean ty 8, A vo ey i . 74 a 5 4 8 8 
iin ete ay Th Cok Karin oe Cad, A ee ae} et. : ; : 
Sale Med Sele) even & A he. So ir eee oe ee) TO iw te Oh ee “ a) . . ‘ es 
Ph RO a Phi € 60g ITN Wee Wy eT A Od Maley Pa. 8 Gers 2 . 
LaLa AE a Sa MO SEEN RU e Fee eek ts ® 
esgfe te it Cera Bre Rae Stee @ giere.e 1) we i ee ta re | S 
RECO Cty) Rukh Kath ote | LA a yt Oreo kh ay y a) Y 
Che Tee a he Pe al RA aN UR roe LO ear) ) . 3 ° 
igo oe ae apt ey Ct i Nee Feb 6S SONOMA ne oo Y 
ats nals ete ST tea eS NR OT itt ee ee ee 
asia “petty on bh TC re e. Bx RR * f 
, u i bd | AR Ly m iy * 
TN al a ee ae. oe fet ! 3 ee : Sa vr. ; 
Lao Ny Carr | ss bs ; y 
a : ak jee ‘ A , ps er ee | e aC 
e . 
aay rn im Oe ‘ ae ARG rr a 
H o \ A ' L &« 8 ry q a MY 
4 rE RS J ( , ’ oe Tio a er) , a ‘ &o,s S 
an tance . 5 alee a ee é A vor ’ 
’ J aa & 4 
SS Ro ee : ee 
t oe f Oe en a 3 Pt Sat reas aby : =e : 
ot vag ya earns 4 y rae OY oe eae Ree Be A eye bye - ie 
AOL at YL aster A Y Me ys i tee 1 ¢ ni A 
LRH Rae y eb Ce | oa) ee . wi Le 
ie RR me es tie ok Pi | a 7 5 Ro 
er) qedauasia are ie We Se i n Tee Ai he Hs - 
A tag bee “ok eS » 4 





